11institutetext: ∗Computer Vision Center, Universitat Autònoma de Barcelona,
Spain
{lkang, priba, yaxing, marcal<EMAIL_ADDRESS>
†omni:us, Berlin, Germany
{lei<EMAIL_ADDRESS>

# GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images

Lei Kang∗† Pau Riba∗ Yaxing Wang∗ Marçal Rusiñol∗ Alicia Fornés∗ Mauricio
Villegas†

###### Abstract

Although current image generation methods have reached impressive quality
levels, they are still unable to produce plausible yet diverse images of
handwritten words. On the contrary, when writing by hand, a great variability
is observed across different writers, and even when analyzing words scribbled
by the same individual, involuntary variations are conspicuous. In this work,
we take a step closer to producing realistic and varied artificially rendered
handwriting. We propose a novel method that is able to produce credible
handwritten word images by conditioning the generative process with both
calligraphic style features and textual content. Our generator is guided by
three complementary learning objectives: to produce realistic images, to
imitate a certain handwriting style and to convey a specific textual content.
Our model is unconstrained to any predefined vocabulary, being able to render
whatever input word. Given a sample writer, it is also able to mimic its
calligraphic features in a few-shot setup. We significantly advance over prior
art and demonstrate with qualitative, quantitative and human-based evaluations
the realistic aspect of our synthetically produced images.

###### Keywords:

Generative adversarial networks, style and content conditioning, handwritten
word images.

##  1 Introduction

Few years after the conception of Generative Adversarial Networks (GANs) [12],
we have witnessed an impressive progress on generating illusory plausible
images. From the early low-resolution and hazy results, the quality of the
artificially generated images has been notably enhanced. We are now able to
synthetically produce high-resolution [5] artificial images that are
indiscernible from real ones to the human observer [24]. In the original GAN
architecture, inputs were randomly sampled from a latent space, so that it was
hard to control which kind of images were being generated. The conception of
conditional Generative Adversarial Networks (cGANs) [35] led to an important
improvement. By allowing to condition the generative process on an input class
label, the networks were then able to produce images from different given
types [7]. However, such classes had to be predefined beforehand during the
training stage and thus, it was impossible to produce images from other unseen
classes during inference.

|  |  |  |
---|---|---|---|---
|  |  |  |
|  |  |  |
Figure 1: Turing’s test. Just five of the above words are real. Try to
distinguish them from the artificially generated samples111 The real words
are: "that", "vision", "asked", "hits" and "writer". .

But generative networks have not exclusively been used to produce synthetic
images. The generation of data that is sequential in nature has also been
largely explored in the literature. Generative methods have been proposed to
produce audio signals [9], natural language excerpts [45], video streams [42]
or stroke sequences [13, 15, 11, 48] able to trace sketches, drawings or
handwritten text. In all of those approaches, in order to generate sequential
data, the use of Recurrent Neural Networks (RNNs) has been adopted.

Yet, for the specific case of generating handwritten text, one could also
envisage the option of directly producing the final images instead of
generating the stroke sequences needed to pencil a particular word. Such non-
recurrent approach presents several benefits. First, the training procedure is
more efficient since recurrencies are avoided and the inherent parallelism
nature of convolutional networks is leveraged. Second, since the output is
generated all at once, we avoid the difficulties of learning long-range
dependencies as well as vanishing gradient problems. Finally, online training
data (pen-tip location sequences), which is hard to obtain, is no longer
needed.

Nevertheless, the different attempts to directly generate raw word images
present an important drawback. Similarly to the case with cGANs, most of the
proposed approaches are just able to condition the word image generation to a
predefined set of words, limiting its practical use. For example [14] is
specifically designed to generate isolated digits, while [6] is restricted to
a handful of Chinese characters. To our best knowledge, the only exception to
that is the approach by Alonso _et al._ [1]. In their work they propose a non-
recurrent generative architecture conditioned to input content strings. By
having such design, the generative process is not restricted to a particular
predefined vocabulary, and could potentially generate any word. However, the
produced results are not realistic, still exhibiting a poor quality, sometimes
producing barely legible word images. Their proposed approach also suffers
from the mode collapse problem, tending to produce images with a unique
writing style. In this paper we present a non-recurrent generative
architecture conditioned to textual content sequences, that is specially
tailored to produce realistic handwritten word images, indistinguishable to
humans. Real and generated images are actually difficult to tell apart, as
shown in Fig. 1. In order to produce diverse styled word images, we propose to
condition the generative process not only with textual content, but also with
a specific writing style, defined by a latent set of calligraphic attributes.

Therefore, our approach111Our code is available at https://github.com/omni-
us/research-GANwriting is able to artificially render realistic handwritten
word images that match a certain textual content and that mimic some style
features (text skew, slant, roundness, stroke width, ligatures, etc.) from an
exemplar writer. To this end, we guide the learning process by three different
learning objectives [36]. First, an adversarial discriminator ensures that the
images are realistic and that its visual appearance is as closest as possible
to real handwritten word images. Second, a style classifier guarantees that
the provided calligraphic attributes, characterizing a particular handwriting
style, are properly transferred to the generated word instances. Finally, a
state-of-the-art sequence-to-sequence handwritten word recognizer [34]
controls that the textual contents have been properly conveyed during the
image generation. To summarize, the main contributions of the paper are the
following:

  * •

Our model conditions the handwritten word generative process both with
calligraphic style features and textual content, producing varied samples
indistinguishable by humans, surpassing the quality of the current state-of-
the-art approaches.

  * •

We introduce the use of three complementary learning objectives to guide
different aspects of the generative process.

  * •

We propose a character-based content conditioning that allows to generate any
word, without being restricted to a specific vocabulary.

  * •

We put forward a few-shot calligraphic style conditioning to avoid the mode
collapse problem.

##  2 Related Work

The generation of realistic synthetic handwritten word images is a challenging
task. To this day, the most convincing approaches involved an expensive manual
intervention aimed at clipping individual characters or glyphs [43, 26, 28,
40, 16]. When such approaches were combined with appropriate rendering
techniques including ligatures among strokes, textures and background
blending, the obtained results were indeed impressive. Haines _et al._ [16]
illustrated how such approaches could artificially generate indistinguishable
manuscript excerpts as if they were written by Sir Arthur Conan Doyle, Abraham
Lincoln or Frida Kahlo. Of course such manual intervention is extremely
expensive, and in order to produce large volumes of manufactured images the
use of truetype electronic fonts has also been explored [27, 23]. Although
such approaches benefit from a greater scalability, the realism of the
generated images clearly deteriorates.

With the advent of deep learning, the generation of handwritten text was
approached differently. As shown in the seminal work by Alex Graves [13],
given a reasonable amount of training data, an RNN could learn meaningful
latent spaces that encode realistic writing styles and their variations, and
then generate stroke sequences that trace a certain text string. However, such
sequential approaches [13, 15, 11, 48] need temporal data, obtained by
recording with a digital stylus pen real handwritten samples, stroke-by-
stroke, in vector form.

Contrary to sequential approaches, non-recurrent generative methods have been
proposed to directly produce images. Both variational auto-encoders [25] and
GANs [12] were able to learn the MNIST manifold and generate artificial
handwritten digit images in the original publications. With the emergence of
cGANs [35], able to condition the generative process on an input image rather
than a random noise vector, the adversarial-guided image-to-image translation
problem started to rise. Image-to-image translation has since been applied to
many different style transfer applications, as demonstrated in [21] with the
_pix2pix_ network. Since then, image translation approaches have been
acquiring the ability to disentangle style attributes from the contents of the
input images, producing better style transfer results [39, 37]. Geometry-aware
synthesizing methods [47, 46] have been successfully applied on scene text
images, but cursive words are not considered.

Concerning the generation of handwritten text, such approaches have been
mainly used for synthesising Chinese ideograms [30, 41, 6, 22, 44] and glyphs
[2]. However, they are restricted to a predefined set of content classes. The
incapability to generate out of vocabulary (OOV) text limits its practical
application. Few works can actually deal with OOV words. First, in the work by
Alonso _et al._ [1], the generation of handwritten word samples is conditioned
by character sequences, but it suffers from the mode collapse problem,
hindering the diversity of the generated images. Second, Fogel  _et al._ [10]
generate handwritten word by assembling the images generated by its
characters, but the generated characters have the same receptive field width,
which can make the generated words look unrealistic. Third, Mayr  _et al._
[33] propose a conversion model to approximate online handwriting from offline
data and then apply style transfer method to online data, so that offline
handwritten text images could be generated by leveraging online handwriting
synthesizer. However, this method highly depends on the performance of the
conversion model and needs online data to train. Techniques like FUNIT [29],
able to transfer unseen target styles to the content generated images could be
beneficial for this limitation. In particular, the use of Adaptive Instance
Normalization (AdaIN) layers, proposed in [18], shall allow to align both
textual content and style attributes within the generative process.

Summarizing, state-of-the-art generative methods are still unable to produce
plausible yet diverse images of whatever handwritten word automatically. In
this paper we propose to condition a generative model for handwritten words
with unconstrained text sequences and stylistic typographic attributes, so
that we are able to generate any word with a great diversity over the produced
results.

##  3 Conditioned Handwritten Generation

###  3.1 Problem Formulation

Let $\\{\mathcal{X},\mathcal{Y},\mathcal{W}\\}$ be a multi-writer handwritten
word dataset, containing grayscale word images $\mathcal{X}$, their
corresponding transcription strings $\mathcal{Y}$ and their writer identifiers
$\mathcal{W}=\\{w_{i}\\}_{i=1}^{N}$. Let
$X_{i}=\\{x_{w_{i},j}\\}_{j=1}^{K}\subset\mathcal{X}$ be a subset of $K$
randomly sampled handwritten word images from the same given writer
$w_{i}\in\mathcal{W}$. Let $\mathcal{A}$ be the alphabet containing the
allowed characters (letters, digits, punctuation signs, etc.),
$\mathcal{A}^{l}$ being all the possible text strings with length $l$. Given a
set of images $X_{i}$ as a few-shot example of the calligraphic style
attributes for writer $w_{i}$ on the one hand, and given a textual content
provided by any text string $t\in\mathcal{A}^{l}$ on the other hand; the
proposed generative model has the ability to combine both sources of
information. It has the objective to yield a handwritten word image having
textual content equal to $t$ and sharing calligraphic style attributes with
writer $w_{i}$. Following this formulation, the generative model $H$ is
defined as

$\bar{x}=H\left(t,X_{i}\right)=H\left(t,\left\\{x_{1},\ldots,x_{K}\right\\}\right),$
(1)

where $\bar{x}$ is the artificially generated handwritten word image with the
desired properties. Moreover, we denote $\bar{\mathcal{X}}$ as the output
distribution of the generative network $H$.

The proposed architecture is divided in two main components. The generative
network produces human-readable images conditioned to the combination of
calligraphic style and textual content information. The second component are
the learning objectives which guide the generative process towards producing
images that look realistic; exhibiting a particular calligraphic style
attributes; and having a specific textual content. Fig. 2 gives an overview of
our model.

Figure 2: Architecture of the proposed handwriting generation model.

###  3.2 Generative Network

The proposed generative architecture $H$ consists of a calligraphic style
encoder $S$, a textual content encoder $C$ and a conditioned image generator
$G$. The overall calligraphic style of input images $X_{i}$ is disentangled
from their individual textual contents, whereas the string $t$ provides the
desired content.

Calligraphic style encoding. Given the set $X_{i}\subset\mathcal{X}$ of $K=15$
word images from the same writer $w_{i}$, the style encoder aims at extracting
the calligraphic style attributes, _i.e._ slant, glyph shapes, stroke width,
character roundness, ligatures etc. from the provided input samples.
Specifically, our proposed network $S$ learns a style latent space mapping, in
which the obtained style representations $F_{s}=S(X_{i})$ are disentangled
from the actual textual contents of the images $X_{i}$. The VGG-19-BN [38]
architecture is used as the backbone of $S$. In order to process the input
image set $X_{i}$, all the images are resized to have the same height $h$,
padded to meet a maximum width $w$ and concatenated channel-wise to end up
with a single tensor $h\times w\times K$. If we ask a human to write the same
word several times, slight involuntary variations appear. In order to imitate
this phenomenon, randomly choosing permutations of the subset $X_{i}$ will
already produce such characteristic fluctuations. In addition, an additive
noise $Z\sim\mathcal{N}(0,1)$ is applied to the output latent space to obtain
a subtly distorted feature representation $\hat{F_{s}}=F_{s}+Z$.

Textual content encoding. The textual content network $C$ is devoted to
produce an encoding of the given text string $t$ that we want to artificially
write. The proposed architecture outputs content features at two different
levels. Low-level features encode the different characters that form a word
and their spatial position within the string. A subsequent broader
representation aims at guiding the whole word consistency. Formally, let
$t\in\mathcal{A}^{l}$ be the input text string, character sequences shorter
than $l$ are padded with the empty symbol $\varepsilon$. Let us define a
character-wise embedding function $e\colon\mathcal{A}\to\mathbb{R}^{n}$. The
first step of the content encoding stage embeds with a linear layer each
character $c\in t$, represented by a one-hot vector, into a character-wise
latent space. Then, the architecture is divided into two branches.

_Character-wise encoding:_ Let $g_{1}\colon\mathbb{R}^{n}\to\mathbb{R}^{m}$ be
a Multi-Layer Perceptron (MLP). Each embedded character $e(c)$ is processed
individually by $g_{1}$ and their results are later stacked together. In order
to combine such representation with style features, we have to ensure that the
content feature map meets the shape of $\hat{F_{s}}$. Each character embedding
is repeated multiple times horizontally to coarsely align the content features
with the visual ones extracted from the style network, and the tensor is
finally vertically expanded. The two feature representations are concatenated
to be fed to the generator $F=[\hat{F_{s}}\parallel F_{c}]$. Such a character-
wise encoding enables the network to produce OOV words, _i.e._ words that have
never been seen during training.

_Global string encoding:_ Let $g_{2}\colon\mathbb{R}^{l\cdot
n}\to\mathbb{R}^{2p\cdot q}$ be another MLP aimed at obtaining a much broader
and global string representation. The character embeddings $e(c)$ are
concatenated into a large one-dimensional vector of size $l\cdot n$ that is
then processed by $g_{2}$. Such global representation vector $f_{c}$ will be
then injected into the generator splitted into $p$ pairs of parameters.

Both functions $g_{1}(\cdot)$ and $g_{2}(\cdot)$ make use of three fully-
connected layers with ReLU activation functions and batch normalization [20].

Generator. Let $F$ be the combination of the calligraphic style attributes and
the textual content information character-wise; and $f_{c}$ the global textual
encoding. The generator $G$ is composed of two residual blocks [19] using the
AdaIN as the normalization layer. Then, four convolutional modules with
nearest neighbor up-sampling and a final $\tanh$ activation layer generates
the output image $\bar{x}$. AdaIN is formally defined as

$\operatorname{AdaIN}\left(z,\alpha,\beta\right)=\alpha\left(\frac{z-\mu\left(z\right)}{\sigma\left(z\right)}\right)+\beta,$
(2)

where $z\in F$, $\mu$ and $\sigma$ are the channel-wise mean and standard
deviations. The global content information is injected four times ($p=4$)
during the generative process by the AdaIN layers. Their parameters $\alpha$
and $\beta$ are obtained by splitting $f_{c}$ in four pairs. Hence, the
generative network is defined as

$\bar{x}=H\left(t,X_{i}\right)=G\left(C\left(t\right),S\left(X_{i}\right)\right)=G\left(g_{1}\left(\hat{t}\right),g_{2}\left(\hat{t}\right),S\left(X_{i}\right)\right),$
(3)

where $\hat{t}=\left[e(c);\forall c\in t\right]$ is the encoding of the string
$t$ character by character.

###  3.3 Learning Objectives

We propose to combine three complementary learning objectives: a
discriminative loss, a style classification loss and a textual content loss.
Each one of these losses aim at enforcing different properties of the desired
generated image $\bar{x}$.

Discriminative Loss. Following the paradigm of GANs [12], we make use of a
discriminative model $D$ to estimate the probability that samples come from a
real source, _i.e._ training data $\mathcal{X}$, or belong to the artificially
generated distribution $\bar{\mathcal{X}}$. Taking the generative network $H$
and the discriminator $D$, this setting corresponds to a $\min\max$
optimization problem. The proposed discriminator $D$ starts with a
convolutional layer, followed by six residual blocks with LeakyReLU
activations and average poolings. A final binary classification layer is used
to discern between fake and real images. Thus, the discriminative loss only
controls that the general visual appearance of the generated image looks
realistic. However, it does not take into consideration neither the
calligraphic styles nor the textual contents. This loss is formally defined as

$\mathcal{L}_{d}\left(H,D\right)=\mathbb{E}_{x\sim\mathcal{X}}\left[\log\left(D\left(x\right)\right)\right]+\mathbb{E}_{\bar{x}\sim\bar{\mathcal{X}}}\left[\log\left(1-D\left(\bar{x}\right)\right)\right].$
(4)

Style Loss. When generating realistic handwritten word images, encoding
information related to calligraphic styles not only provides diversity on the
generated samples, but also prevents the mode collapse problem. Calligraphy is
a strong identifier of different writers. In that sense, the proposed style
loss guides the generative network $H$ to generate samples conditioned to a
particular writing style by means of a writer classifier $W$. Given a
handwritten word image, $W$ tries to identify the writer $w_{i}\in\mathcal{W}$
who produced it. The writer classifier $W$ follows the same architecture of
the discriminator $D$ with a final classification MLP with the amount of
writers in our training dataset. The classifier $W$ is only optimized with
real samples drawn from $\mathcal{X}$, but it is used to guide the generation
of the synthetic ones. We use the cross entropy loss, formally defined as

$\mathcal{L}_{w}\left(H,W\right)=-\mathbb{E}_{x\sim\left\\{\mathcal{X},\bar{\mathcal{X}}\right\\}}\left[\sum_{i=1}^{\left|\mathcal{W}\right|}w_{i}\log\left(\hat{w}_{i}\right)\right],$
(5)

where $\hat{w}=W(x)$ is the predicted probability distribution over writers in
$\mathcal{W}$ and $w_{i}$ the real writer distribution. Generated samples
should be classified as the writer $w_{i}$ used to construct the input style
conditioning image set $X_{i}$.

Figure 3: Architecture of the attention-based sequence-to-sequence handwritten
word recognizer $R$.

Content Loss. A final handwritten word recognizer network $R$ is used to guide
our generator towards producing synthetic word images with a specific textual
content. We implemented a state-of-the-art sequence-to-sequence model [34] for
handwritten word recognition to examine whether the produced images $\bar{x}$
are actually decoded as the string $t$. The recognizer, depicted in Fig. 3,
consists of an encoder and a decoder coupled with an attention mechanism.
Handwritten word images are processed by the encoder and high-level feature
representations are obtained. A VGG-19-BN [38] architecture followed by a two-
layered Bi-directional Gated Recurrent Unit (B-GRU) [8] is used as the encoder
network. The decoder is a one-directional RNN that outputs character by
character predictions at each time step. The attention mechanism dynamically
aligns context features from each time step of the decoder with high-level
features from the encoder, hopefully corresponding to the next character to
decode. The Kullback-Leibler divergence loss is used as the recognition loss
at each time step. This is formally defined as

$\mathcal{L}_{r}\left(H,R\right)=-\mathbb{E}_{x\sim\left\\{\mathcal{X},\bar{\mathcal{X}}\right\\}}\left[\sum_{i=0}^{l}\sum_{j=0}^{\left|\mathcal{A}\right|}t_{i,j}\log\left(\frac{t_{i,j}}{\hat{t}_{i,j}}\right)\right],$
(6)

where $\hat{t}=R(x)$; $\hat{t}_{i}$ being the $i$-th decoded character
probability distribution by the word recognizer, $\hat{t}_{i,j}$being the
probability of $j$-th symbol in $\mathcal{A}$ for $\hat{t}_{i}$, and $t_{i,j}$
being the real probability corresponding to $\hat{t}_{i,j}$. The empty symbol
$\varepsilon$ is ignored in the loss computation; $t_{i}$ denotes the $i$-th
character on the input text $t$.

###  3.4 End-to-end Training

Overall, the whole architecture is trained end to end with the combination of
the three proposed loss functions

$\mathcal{L}(H,D,W,R)=\mathcal{L}_{d}(H,D)+\mathcal{L}_{w}(H,W)+\mathcal{L}_{r}(H,R),$
(7) $\min_{H,W,R}\max_{D}\mathcal{L}(H,D,W,R).$ (8)

Algorithm 1 presents the training strategy that has been followed in this
work. $\Gamma(\cdot)$ denotes the optimizer function. Note that the parameter
optimization is performed in two steps. First, the discriminative loss is
computed using both real and generated samples (line 3). The style and content
losses are computed by just providing real data (line 4). Even though $W$ and
$D$ are optimized using only real data and, therefore, they could be pre-
trained independently from the generative network $H$, we obtained better
results by initializing all the networks from scratch and jointly training
them altogether. The network parameters $\Theta_{D}$ are optimized by gradient
ascent following the GAN paradigm whereas the parameters $\Theta_{W}$ and
$\Theta_{R}$ are optimized by gradient descent. Finally, the overall generator
loss is computed following Equation 7 where only the generator parameters
$\Theta_{H}$ are optimized (line 8).

Input: Input data $\\{\mathcal{X},\mathcal{Y},\mathcal{W}\\}$; alphabet
$\mathcal{A}$; max training iterations $T$
Output:  Networks parameters
$\\{\Theta_{H},\Theta_{D},\Theta_{W},\Theta_{R}\\}$.

1:repeat

2: Get style and content mini-batches $\\{X_{i},w_{i}\\}_{i=1}^{N_{B}}$ and
$\\{t^{i}\\}_{i=1}^{N_{B}}$

3: $\mathcal{L}_{d}\leftarrow$ Eq. 4 $\triangleright$ Real and generated
samples $x\sim\\{\mathcal{X},\bar{\mathcal{X}}\\}$

4: $\mathcal{L}_{w,r}\leftarrow$ Eq. 5 \+ Eq. 6 $\triangleright$ Real samples
$x\sim\mathcal{X}$

5: $\Theta_{D}\leftarrow\Theta_{D}+\Gamma(\nabla_{\Theta_{D}}\mathcal{L}_{d})$

6:
$\Theta_{W,R}\leftarrow\Theta_{W,R}-\Gamma(\nabla_{\Theta_{W,R}}\mathcal{L}_{w,d})$

7: $\mathcal{L}\leftarrow$ Eq. 7 $\triangleright$ Generated samples
$x\sim\bar{\mathcal{X}}$

8: $\Theta_{H}\leftarrow\Theta_{H}-\Gamma(\nabla_{\Theta_{H}}\mathcal{L})$

9:until Max training iterations $T$

Algorithm 1 Training algorithm for the proposed model.

##  4 Experiments

To carry out the different experiments, we have used a subset of the IAM
corpus [32] as our multi-writer handwritten dataset
$\\{\mathcal{X},\mathcal{Y},\mathcal{W}\\}$. It consists of $62,857$
handwritten word snippets, written by 500 different individuals. Each word
image has its associated writer and transcription metadata. A test subset of
160 writers has been kept apart during training to check whether the
generative model is able to cope with unseen calligraphic styles. We have also
used a subset of $22,500$ unique English words from the Brown [3] corpus as
the source of strings for the content input. A test set of 400 unique words,
disjoint from the IAM transcriptions has been used to test the performance
when producing OOV words. To quantitatively measure the image quality,
diversity and the ability to transfer style attributes of the proposed
approach we will use the Fréchet Inception Distance (FID) [17, 4], measuring
the distance between the Inception-v3 activation distributions for generated
$\bar{\mathcal{X}}$ and real samples $\mathcal{X}$ for each writer $w_{i}$
separately, and finally averaging them. Inception features, trained over
ImageNet data, have not been designed to discern between different handwriting
images. Although this measure might not be ideal to evaluate our specific
case, it will still serve as an indication of the similarity between generated
and real images.

a) IV-S |  |  |  |  |  |  |
---|---|---|---|---|---|---|---
b) IV-U |  |  |  |  |  |  |
c) OOV-S |  |  |  |  |  |  |
d) OOV-U |  |  |  |  |  |  |
Figure 4: Word image generation. a) In-Vocabulary (IV) words and seen (S)
styles; b) In-Vocabulary (IV) words and unseen (U) styles; c) Out-of-
Vocabulary (OOV) words and seen (S) styles and d) Out-of-Vocabulary (OOV)
words and unseen (U) styles.

###  4.1 Generating Handwritten Word Images

We present in Fig. 4 an illustrative selection of generated handwritten words.
We appreciate the realistic and diverse aspect of the produced images.
Qualitatively, we observe that the proposed approach is able to yield
satisfactory results even when dealing with both words and calligraphic styles
never seen during training. But, when analyzing the different experimental
settings in Table 1, we appreciate that the FID measure slightly degrades when
either we input an OOV word or a style never seen during training.
Nevertheless, the reached FID measures in all four settings satisfactorily
compare with the baseline achieved by real data.

Table 1: FID between generated images and real images of corresponding set. | Real images | IV-S | IV-U | OOV-S | OOV-U
---|---|---|---|---|---
FID | $90.43$ | $120.07$ | $124.30$ | $125.87$ | $130.68$
Figure 5: t-SNE embedding visualization of $2.500$ generated instances of the
word "deep".

In order to show the ability of the proposed method to produce a diverse set
of generated images, we present in Fig. 5 a t-SNE [31] visualization of
different instances produced with a fixed textual content while varying the
calligraphic style inputs. Different clusters corresponding to particular
slants, stroke widths, character roundnesses, ligatures and cursive writings
are observed.

|  | Style Images
---|---|---
|  |  |  |  |  |  |
Textual Content | FUNIT |  |  |  |  |  |
ours |  |  |  |  |  |
"which"
FUNIT |  |  |  |  |  |
ours |  |  |  |  |  |
"Thank"
FUNIT |  |  |  |  |  |
ours |  |  |  |  |  |
"inside"
Figure 6: Comparison of handwritten word generation with FUNIT [29].

To further demonstrate the ability of the proposed approach to coalesce
content and style information into the generated handwritten word images, we
compare in Fig. 6 our produced results with the outcomes of the state-of-the-
art approach FUNIT [29]. Being an image-to-image translation method, FUNIT
starts with a content image and then injects the style attributes derived from
a second sample image. Although FUNIT performs well for natural scene images,
it is clear that such kind of approaches do not apply well for the specific
case of handwritten words. Starting with a content image instead of a text
string confines the generative process to the shapes of the initial drawing.
When infusing the style features, the FUNIT method is only able to deform the
stroke textures, often resulting in extremely distorted words. Conversely, our
proposed generative process is able to produce realistic and diverse word
samples given a content text string and a calligraphic style example. We
observe how for the different produced versions of the same word, the proposed
approach is able to change style attributes as stroke width or slant, to
produce both cursive words, where all characters are connected through
ligatures as well as disconnected writings, and even render the same
characters differently, _e.g._ note the characters n or s in "Thank" or
"inside" respectively.

###  4.2 Latent Space Interpolations

The generator network $G$ learns to map feature points $F$ in the latent space
to synthetic handwritten word images. Such latent space presents a structure
worth exploring. We first interpolate in Fig. 7 between two different points
$F_{s}^{A}$ and $F_{s}^{B}$ corresponding to two different calligraphic styles
$w_{A}$ and $w_{B}$ while keeping the textual contents $t$ fixed. We observe
how the generated images smoothly adjust from one style to another. Again note
how individual characters evolve from one typography to another, _e.g._ the l
from "also", or the f from "final".

| $w_{A}$ |  |  |  |  |  |  |  |  |  | $w_{B}$
---|---|---|---|---|---|---|---|---|---|---|---
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |
Real |  |  |  |  |  |  |  |  |  |  |
Generated |  |  |  |  |  |  |  |  |  |  |

Figure 7: Latent space interpolation between two calligraphic styles for
different words while keeping contents fixed.

Contrary to the continuous nature of the style latent space, the original
content space is discrete in nature. Instead of computing point-wise
interpolations, we present in Fig. 8 the obtained word images for different
styles when following a “word ladder” puzzle game, _i.e._ going from one word
to another, one character difference at a time. Here we observe how different
contents influence stylistic aspects. Usually s and i are disconnected when
rendering the word "sired" but often appear with a ligature when jumping to
the word "fired".

"three" | "threw" | "shrew" | "shred" | "sired" | "fired" | "fined" | "firer" | "fiver" | "fever" | "sever" | "seven"
---|---|---|---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |
Figure 8: Word ladder. From "three" to "seven" changing one character at a
time, generated for five different calligraphic styles.

###  4.3 Impact of the Learning Objectives

Along this paper, we have proposed to guide the generation process by three
complementary goals. The discriminator loss $\mathcal{L}_{d}$ controlling the
genuine appearance of the generated images $\bar{x}$. The writer
classification loss $\mathcal{L}_{w}$ forcing $\bar{x}$ to mimic the
calligraphic style of input images $X_{i}$. The recognition loss
$\mathcal{L}_{r}$ guaranteeing that $\bar{x}$ is readable and conveys the
exact text information $t$. We analyze in Table 2 the effect of each learning
objective.

Table 2: Effect of each different learning objectives when generating the content $t=\texttt{"vision"}$ for different styles. $\mathcal{L}_{d}$ | $\mathcal{L}_{w}$ | $\mathcal{L}_{r}$ | FID | Style Images
---|---|---|---|---
|  |  |
✓ | - | - | 364.10 |  |  |  |
✓ | ✓ | - | 207.47 |  |  |  |
✓ | - | ✓ | 138.80 |  |  |  |
✓ | ✓ | ✓ | 130.68 |  |  |  |

The sole use of the $\mathcal{L}_{d}$ leads to constantly generating an image
that is able to fool the discriminator. Although the generated image looks
like handwritten strokes, the content and style inputs are ignored. When
combining the discriminator with the auxiliary task of writer classification
$\mathcal{L}_{w}$, the produced results are more encouraging, but the input
text is still ignored, always tending to generate the word "the", since it is
the most common word seen during training. When combining the discriminator
with the word recognizer loss $\mathcal{L}_{r}$, the desired word is rendered.
However, as in [1], we suffer from the mode collapse problem, always producing
unvarying word instances. When combining the three learning objectives we
appreciate that we are able to correctly render the appropriate textual
content while mimicking the input styles, producing diverse results. We
appreciate that the FID measure also decreases for each successive
combination.

###  4.4 Human Evaluation

Finally, we also tested whether the generated images were actually
indistinguishable from real ones by human judgments. We have conducted a human
evaluation study as follows: we have asked $200$ human examiners to assess
whether a set of images were written by a human or artificially generated.
Appraisers were presented a total of sixty images, one at a time, and they had
to choose if each of them was real of fake. We chose thirty real words from
the IAM test partition from ten different writers. We then generated thirty
artificial samples by using OOV textual contents and by randomly taking the
previous writers as the sources for the calligraphic styles. Such sets were
not curated, so the only filter was that the generated samples had to be
correctly transcribed by the word recognizer network $R$. In total we
collected $12,000$ responses. In Table 3 we present the confusion matrix of
the human assessments, with Accuracy (ACC), Precision (P), Recall (R), False
Positive Rate (FPR) and False Omission Rate (FOR) values. The study revealed
that our generative model was clearly perceived as plausible, since a great
portion of the generated samples were deemed genuine. Only a $49.3\%$ of the
images were properly identified, which shows a similar performance than a
random binary classifier. Accuracies over different examiners were normally
distributed. We also observe that the synthetically generated word images were
judged more often as being real than correctly identified as fraudulent, with
a final FPR of $55.4\%$.

Table 3: Human evaluation plausibility experiment. | Actual | Predicted |
---|---|---
Real | Fake |
Genuine | $27.01$ | $22.99$ | R: $54.1$
Generated | $27.69$ | $22.31$ | FPR: $55.4$
| P: $49.4$ | FOR: $50.8$ | ACC: $49.3$
a) Confusion matrix ($\%$) | b) Accuracy distribution

##  5 Conclusion

We have presented a novel image generation architecture that produces
realistic and varied artificially rendered samples of handwritten words. Our
pipeline can yield credible word images by conditioning the generative process
with both calligraphic style features and textual content. Furthermore, by
jointly guiding our generator with three different cues: a discriminator, a
style classifier and a content recognizer, our model is able to render any
input word, not depending on any predefined vocabulary, while incorporating
calligraphic styles in a few-shot setup. Experimental results demonstrate that
the proposed method yields images with such a great realistic quality that are
indistinguishable by humans.

## Acknowledgements

This work was supported by EU H2020 SME Instrument project 849628, the Spanish
projects TIN2017-89779-P and RTI2018-095645-B-C21, and grants 2016-DI-087,
FPU15/06264 and RYC-2014-16831. Titan GPU was donated by NVIDIA.

## References

  * [1] Alonso, E., Moysset, B., Messina, R.: Adversarial generation of handwritten text images conditioned on sequences. In: Proceedings of the International Conference on Document Analysis and Recognition (2019)
  * [2] Azadi, S., Fisher, M., Kim, V.G., Wang, Z., Shechtman, E., Darrell, T.: Multi-content gan for few-shot font style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7564–7573 (2018)
  * [3] Bird, S., Klein, E., Loper, E.: Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc. (2009)
  * [4] Borji, A.: Pros and cons of GAN evaluation measures. Computer Vision and Image Understanding 179, 41–65 (2019)
  * [5] Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: Proceedings of the International Conference on Learning Representations (2019)
  * [6] Chang, B., Zhang, Q., Pan, S., Meng, L.: Generating handwritten Chinese characters using CycleGAN. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (2018)
  * [7] Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
  * [8] Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Proceedings of the NeurIPS Workshop on Deep Learning (2014)
  * [9] Dong, H.W., Hsiao, W.Y., Yang, L.C., Yang, Y.H.: MuseGAN: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)
  * [10] Fogel, S., Averbuch-Elor, H., Cohen, S., Mazor, S., Litman, R.: Scrabblegan: Semi-supervised varying length handwritten text generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4324–4333 (2020)
  * [11] Ganin, Y., Kulkarni, T., Babuschkin, I., Eslami, S., Vinyals, O.: Synthesizing programs for images using reinforced adversarial learning. In: Proceedings of the International Conference on Machine Learning (2018)
  * [12] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proceedings of the Neural Information Processing Systems Conference (2014)
  * [13] Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
  * [14] Gregor, K., Danihelka, I., Graves, A., Rezende, D.J., Wierstra, D.: DRAW: A recurrent neural network for image generation. In: Proceedings of the International Conference on Machine Learning (2015)
  * [15] Ha, D., Eck, D.: A neural representation of sketch drawings. In: Proceedings of the International Conference on Learning Representations (2018)
  * [16] Haines, T.S., Mac Aodha, O., Brostow, G.J.: My text in your handwriting. ACM Transactions on Graphics 35(3), 1–18 (2016)
  * [17] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Proceedings of the Neural Information Processing Systems Conference (2017)
  * [18] Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision (2017)
  * [19] Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Proceedings of the European Conference on Computer Vision (2018)
  * [20] Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning (2015)
  * [21] Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
  * [22] Jiang, H., Yang, G., Huang, K., Zhang, R.: W-net: one-shot arbitrary-style Chinese character generation with deep neural networks. In: Proceedings of the International Conference on Neural Information Processing (2018)
  * [23] Kang, L., Rusiñol, M., Fornés, A., Riba, P., Villegas, M.: Unsupervised adaptation for synthetic-to-real handwritten word recognition. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. pp. 3491–3500 (2020)
  * [24] Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
  * [25] Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: Proceedings of the International Conference on Learning Representations (2014)
  * [26] Konidaris, T., Gatos, B., Ntzios, K., Pratikakis, I., Theodoridis, S., Perantonis, S.J.: Keyword-guided word spotting in historical printed documents using synthetic data and user feedback. International Journal of Document Analysis and Recognition 9(2-4), 167–177 (2007)
  * [27] Krishnan, P., Jawahar, C.: Generating synthetic data for text recognition. arXiv preprint arXiv:1608.04224 (2016)
  * [28] Lin, Z., Wan, L.: Style-preserving English handwriting synthesis. Pattern Recognition 40(7), 2097–2109 (2007)
  * [29] Liu, M.Y., Huang, X., Mallya, A., Karras, T., Aila, T., Lehtinen, J., Kautz, J.: Few-shot unsupervised image-to-image translation. In: Proceedings of the IEEE International Conference on Computer Vision (2019)
  * [30] Lyu, P., Bai, X., Yao, C., Zhu, Z., Huang, T., Liu, W.: Auto-encoder guided GAN for Chinese calligraphy synthesis. In: Proceedings of the International Conference on Document Analysis and Recognition (2017)
  * [31] Maaten, L.v.d., Hinton, G.: Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2579–2605 (2008)
  * [32] Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition 5(1), 39–46 (2002)
  * [33] Mayr, M., Stumpf, M., Nikolaou, A., Seuret, M., Maier, A., Christlein, V.: Spatio-temporal handwriting imitation. arXiv preprint arXiv:2003.10593 (2020)
  * [34] Michael, J., Labahn, R., Grüning, T., Zöllner, J.: Evaluating sequence-to-sequence models for handwritten text recognition. arXiv preprint arXiv:1903.07377 (2019)
  * [35] Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  * [36] Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: Proceedings of the International Conference on Machine Learning (2017)
  * [37] Pondenkandath, V., Alberti, M., Diatta, M., Ingold, R., Liwicki, M.: Historical document synthesis with generative adversarial networks. In: Proceedings of the International Conference on Document Analysis and Recognition (2019)
  * [38] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representations (2015)
  * [39] Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. In: Proceedings of the International Conference on Learning Representations (2017)
  * [40] Thomas, A.O., Rusu, A., Govindaraju, V.: Synthetic handwritten CAPTCHAs. Pattern Recognition 42(12), 3365–3373 (2009)
  * [41] Tian, Y.: zi2zi: Master chinese calligraphy with conditional adversarial networks (2017), https://github.com/kaonashi-tyc/zi2zi
  * [42] Tulyakov, S., Liu, M.Y., Yang, X., Kautz, J.: MoCoGAN: Decomposing motion and content for video generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
  * [43] Wang, J., Wu, C., Xu, Y.Q., Shum, H.Y.: Combining shape and physical models for online cursive handwriting synthesis. International Journal of Document Analysis and Recognition 7(4), 219–227 (2005)
  * [44] Wu, S.J., Yang, C.Y., Hsu, J.Y.j.: Calligan: Style and structure-aware chinese calligraphy character generator. arXiv preprint arXiv:2005.12500 (2020)
  * [45] Yu, L., Zhang, W., Wang, J., Yu, Y.: SeqGAN: Sequence generative adversarial nets with policy gradient. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)
  * [46] Zhan, F., Xue, C., Lu, S.: Ga-dan: Geometry-aware domain adaptation network for scene text detection and recognition. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 9105–9115 (2019)
  * [47] Zhan, F., Zhu, H., Lu, S.: Spatial fusion gan for image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3653–3662 (2019)
  * [48] Zheng, N., Jiang, Y., Huang, D.: Strokenet: A neural painting environment. In: Proceedings of the International Conference on Learning Representations (2019)


# A Combinatorial Proof of a Partition Perimeter Inequality

Hunter Waldron

###### Abstract.

The partition perimeter is a statistic defined to be one less than the sum of
the number of parts and the largest part. Recently, Amdeberhan, Andrews, and
Ballantine proved the following analog of Glaisher’s theorem: for all $m\geq
2$ and $n\geq 1$, there are at least as many partitions with perimeter $n$ and
parts $\not\equiv 0\pmod{m}$ as partitions with perimeter $n$ and parts
repeating fewer than $m$ times. In this work, we provide a combinatorial proof
of their theorem by relating the combinatorics of the partition perimeter to
that of compositions. Using this technique, we also show that a composition
theorem of Huang implies a refinement of another perimeter theorem of Fu and
Tang.

##  1\. Introduction

The partitions of an integer $n\geq 1$, which are finite sequences of integers
$\lambda=(\lambda_{1},\lambda_{2},\dots,\lambda_{\ell})$ where the parts
$\lambda_{i}$ sum to $n$ and satisfy the inequalities
$\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{\ell}$, have long been
studied for their wealth of enumerative properties. Many seemingly unrelated
families of partitions have been shown to be equinumerous. Euler’s theorem is
a classic example of this kind of result:

######  Theorem 1.1.

For all $n\geq 1$, there are as many partitions of $n$ with odd parts as there
are partitions of $n$ with distinct parts.

The perimeter of a partition is a statistic defined to be the value
$\lambda_{1}+\ell(\lambda)-1$ where $\ell(\lambda)$ is the number of parts in
$\lambda$. This is Straub’s definition [7], which is the most convenient for
the purposes of this work. Many other partition statistics like the rank
$\lambda_{1}-\ell(\lambda)$ allow infinitely many partitions to take on each
value, and so must be studied in addition to partition size $n$. There are
only $2^{k-1}$ partitions with perimeter $k\geq 1$, which makes viable the
study of partitions by their perimeter alone.

Interest in enumeration by perimeter (see [1, 2, 4]) spawned from the
following result of Straub, which is remarkably similar in presentation to
Euler’s theorem.

######  Theorem 1.2 (Theorem 1.4 in [7]).

For all $n\geq 1$, there are as many partitions with perimeter $n$ and odd
parts as there are partitions with perimeter $n$ and distinct parts.

Naturally, for any theorem like Euler’s we can attempt to replace size with
perimeter and show that an analogous result holds. For instance, Glaisher’s
theorem generalizes Euler’s to allow an arbitrary modulus:

######  Theorem 1.3.

For all $m\geq 2$ and $n\geq 1$, there are as many partitions of $n$ with
parts $\not\equiv 0\pmod{m}$ as there are partitions of $n$ with parts
repeating fewer than $m$ times.

Let $h_{m}(n)$ and $g_{m}(n)$ be the sets of partitions with perimeter $n$
having parts $\not\equiv 0\pmod{m}$ and parts repeating fewer than $m$ times,
respectively. In [1], Amdeberhan, Andrews, and Ballantine showed that Straub’s
theorem actually becomes an inequality when generalized in the direction of
Glaisher’s theorem.

######  Theorem 1.4 (Theorem 7.6 in [1]).

We have $h_{m}(n)-g_{m}(n)\geq 0$ for all $m\geq 2$ and $n\geq 1$.

They proved this result by manipulating generating functions, and asked if a
combinatorial proof exists. In this work, we provide such a proof. The
technique we use appears to have additional utility in this kind of research.
As an example, we provide another application to prove a refinement of the
following theorem of Fu and Tang.

######  Theorem 1.5 (Theorem 2.15 in [2]).

For all $m,n\geq 1$, there are as many partitions with parts $\equiv
1\pmod{m+1}$ and perimeter $n$ as there are partitions with gaps between parts
$\geq m$ and perimeter $n$.

Our refinement, which we state here, adds the additional parameter $k$, and
reduces to Fu and Tang’s theorem when $k=0$, and then to Straub’s theorem when
$m=1$:

######  Theorem 1.6.

For all $n,m\geq 1$ and $k\geq 0$, the following families of partitions with
perimeter $n$ are equinumerous:

  * 1)

Partitions $\lambda$ with parts grouped into $k$ non-empty and disjoint sets
of sequential parts $\\{\lambda_{1},\dots,\lambda_{t_{1}}\\}$,
$\\{\lambda_{t_{1}+1},$ $\dots,$ $\lambda_{t_{2}}\\},$ $\dots,$
$\\{\lambda_{t_{k-1}+1},\dots,\lambda_{t_{k}=\ell(\lambda)}\\}$, or $k+1$ sets
if $\lambda_{\ell(\lambda)}\equiv 1\pmod{m+1}$ but with
$\lambda_{\ell(\lambda)}>m+1$ otherwise, where each set has parts belonging
all to the same congruence class modulo $m+1$, and neighboring sets have
different congruence classes and gaps between them
$\lambda_{t_{i}}-\lambda_{t_{i+1}}>m$ for each $1\leq i<k$.

  * 2)

Partitions $\lambda$ with gaps between parts $\geq m$ with exactly $k$
exceptions $\lambda_{i}-\lambda_{i+1}<m$, always proceeded by a gap
$\lambda_{i-1}-\lambda_{i}\geq m$, and followed by a gap
$\lambda_{i+1}-\lambda_{i+2}>m$ unless $\ell(\lambda)=i+1$.

Family 1) | Family 2)
---|---
|
Figure 1. Example of the partitions enumerated in Theorem 1.6 with $m=1$,
$k=1$, and $n=7$.

The rest of this paper is organized into two sections. Section 2 contains a
description of the technique used in the proofs, as well as the proof of
Theorem 1.6. The combinatorial proof of Theorem 1.4 is in section 3, as well
as some commentary.

##  2\. Compositions and the partition perimeter

A composition of the integer $n\geq 1$ is a partition
$c=(c_{1},c_{2},\dots,c_{\ell})$ of $n$ with the requirement that $c_{1}\geq
c_{2}\geq\cdots\geq c_{\ell}$ discarded. We borrow the same terminology and
tools for compositions from partitions when appropriate.

We may visually represent $c$ as a Young diagram, which is a grid of squares
containing $c_{i}$ squares in the $i$th row from the top, aligned on the left.
The $m$-modular diagram of $c$ for fixed $m\geq 2$ is a Young diagram where
the squares furthest to the right contain a value between 1 and $m$, and all
others contain $m$, having the property that the sum of the values in the
$i$th row is $c_{i}$.

323312331 Figure 2. The Young diagram and $3$-modular diagram of
$c=(5,3,4,2,7)$.

For any partition $\lambda$, the conjugate $\lambda^{\prime}$ is a new
partition with $\lambda_{i}^{\prime}$ defined to be the $i$th column of
$\lambda$’s Young diagram, from the left. Conjugation is an involution that
preserves the perimeter since $\lambda_{1}$ and $\ell(\lambda)$ are swapped.

Remark. Conjugation is only defined for partitions, although an analogous
operation exists for compositions in general (See the work of Munagi such as
[6] and the references therein for recent work).

Figure 3. The conjugate partition pair $(5,3,1,1)$ and $(4,2,2,1,1)$.

The perimeter of a partition $\lambda$ can be equivalently defined as the
number of squares with an edge or corner on the bottom or right side of
$\lambda$’s Young diagram. Let

$c=(\lambda_{1}-\lambda_{2}+1,\lambda_{2}-\lambda_{3}+1,\dots,\lambda_{\ell(\lambda)-1}-\lambda_{\ell(\lambda)}+1,\lambda_{\ell(\lambda)}).$

Clearly $c_{i}$ counts the number of squares from the $i$th row of $\lambda$’s
Young diagram that contribute to the perimeter, so $c$ is a composition with
the properties $\lvert c\rvert=\lambda_{1}+\ell(\lambda)-1$ and
$\ell(c)=\ell(\lambda)$. Indeed, this defines a bijection since each part of
$\lambda$ can easily be recovered using the formula

$\lambda_{i}=1+\sum_{j=i}^{\ell(c)}(c_{j}-1).$

Let $\mathcal{C}$ and $\mathcal{P}$ be the sets of all compositions and
partitions, respectively, and $\pi\colon\mathcal{C}\to\mathcal{P}$ be this
bijection. We can also interpret $\pi(c)$ as the partition formed by sliding
the rows of $c$ over so they overlap at exactly one square.

$p(c)$ Figure 4. An example of the bijection $\pi$ with $c=(3,1,2,4,2)$ and
$\pi(c)=(8,6,6,5,2)$.

Using $\pi$ and these Young diagram interpretations, we can encode information
about the partition perimeter into a composition, which allows us to then work
entirely with compositions, only switching back with $\pi^{-1}$ when done.

For an example of how this technique may be used, consider the following
composition result of Munagi.

######  Theorem 2.1 (Theorem 1.2 in [5]).

For all $n,m\geq 1$ there are as many compositions of $n$ with parts $\equiv
1\pmod{m}$ as there are compositions of $n+m-1$ with parts $\geq m$.

Replacing $m$ with $m+1$, the composition parts from the first family become
under $\pi$ partition parts $\equiv 1\pmod{m+1}$ since they all have the
effect of preserving the congruence of the part below. Deleting $m$ from the
second family’s last part, which is guaranteed to be $\geq m+1$, brings the
size down to $n$. Sliding the parts over as in Figure 4, we obtain partitions
with gaps between parts $\geq m$, and so we arrive at Theorem 1.5.

The proof of Theorem 1.6 is similar to this, using the following refinement of
Munagi’s theorem from Huang.

######  Theorem 2.2 (Theorem 1.6 in [3]).

For all $n,m\geq 1$, and $k\geq 0$, there are as many compositions of $n$ with
exactly $k$ parts $\not\equiv 1\pmod{m}$, each $>m$, as there are compositions
of $n+m-1$ with exactly $k$ parts $<m$, each proceeded by a part $\geq m$ and
followed by either a last part or a part $>m$.

Proof of Theorem 1.6. Proceed as above, replacing $m$ with $m+1$ now using
Theorem 2.2.

For family 1), any composition parts $\not\equiv 1\pmod{m+1}$ have the effect
under $\pi$ of changing from the part below’s congruence class, if there is
one, so each of the $k$ sets are formed from a part $\not\equiv 1\pmod{m+1}$,
followed above by any amount of parts $\equiv 1\pmod{m+1}$.

For family 2), delete $m$ from the final part of each composition. Each of the
$k$ parts $<m+1$ along with the guaranteed part above $\geq m+1$ and below
$>m+1$ produce under $\pi$ a gap $\geq m$, $<m$, and then $>m$.

The omitted details are straight forward to verify.

∎

Remark. Huang provides a formula (Theorem 1.7 in [3]) that the partitions in
Theorem 1.6 inherit.

Using this technique in the other direction is possible too, yielding
composition results. Although this idea is not pursued in this work, we
provide the following simple example.

######  Proposition 2.3.

For all $m\geq 2$ and $n,k\geq 1$, there are as many compositions of $n$ with
$k$ parts, all at most $m$ and last part less than $m$, as there are
compositions of $n$ with $n-k+1$ parts and no continuous sequence of $m-1$ or
more $1$’s, excluding the last part.

Proof. Partitions with perimeter $n$, $k$ parts, and gaps between parts $<m$
conjugate to give partitions with perimeter $n$, $n-k+1$ parts since $k-1$
squares in the Young diagram had a perimeter contributing square below them,
and parts repeating $<m$ times. Interpreting these families through $\pi^{-1}$
gives the result.

∎

##  3\. Combinatorial proof of Theorem 1.4

Before proceeding, we need the following lemma. Here we use the notation
$\equiv R\;(\textrm{mod}\;m)$ to mean congruent to some $r\in R$ modulo $m$.

######  Lemma 3.1.

Let $m\geq 2$, $n\geq 1$, and $R\subseteq\\{1,2,\dots,m-1\\}$ be non-empty.
There are as many compositions of $n$ with parts $\equiv R\;(\textrm{mod}\;m)$
as there are compositions of $n$ with parts in $R\cup\\{m\\}$, with last part
in $R$.

Proof. Let $c$ be a composition with parts $\equiv R\;(\textrm{mod}\;m)$.
Writing $c$ as an $m$-modular diagram, rotate each part one quarter turn
clockwise, leaving the $m$-modular diagram of a composition with parts in
$R\cup\\{m\\}$ and last part in $R$. Since the numbers were unchanged, this
resulting composition has the same size.

For the inverse, given a composition $c$ with parts in $R\cup\\{m\\}$ and last
part in $R$, iterate upwards in the $m$-modular diagram starting from $c$’s
last part. Collect the square not containing $m$ and then all squares
containing $m$ directly above, then rotate counter-clockwise to form a part
$\equiv R\;(\mathrm{mod}\;m)$. Repeat until all squares are exhausted.

∎

441433431434411433 Figure 5.  With $m=4$ and $R=\\{1,3\\}$, the composition
$(7,9,1,7,3)$ under this bijection maps to $(4,3,4,4,1,1,4,3,3)$.

Combinatorial proof of Theorem 1.4. Fix the integers $m$ and $n$. The proof is
built using a series of simple bijections and one injection between the
following six families of compositions and partitions.

  * 1)

Partitions with perimeter $n$ and parts $\not\equiv 0\pmod{m}$ become

  * 2)

compositions $c$ of $n$ with the property that

($\ast$) $1+\sum_{j=i}^{\ell(c)}(c_{j}-1)\not\equiv
0\;(\mathrm{mod}\;m)\;\mathrm{for\;all}\;1\leq i\leq\ell(c)$

through $\pi^{-1}$. The injection $\varphi$ defined below sends these to

  * 3)

compositions of $n$ with parts $\not\equiv 0\pmod{m}$. Next, using Lemma 3.1
with $R=\\{1,\dots,m-1\\}$, these become

  * 4)

compositions of $n$ with all parts $\leq m$ and last part $<m$. Applying
$\pi$, we then get

  * 5)

partitions with perimeter $n$ and gaps between parts $<m$ and then finally

  * 6)

partitions with perimeter $n$ and parts repeating $<m$ times, by conjugation.

The injection $\varphi$ is defined as follows. Let $c$ be a composition with
the property ($\ast$ ‣ 2)). From $c$ we construct $\varphi(c)=d$ using the
following iterative algorithm, starting at $c$’s last part. The algorithm is
designed to preserve parts in $c$ that are already $\not\equiv
0\;(\mathrm{mod}\;m)$, and transform the others in such a way that a part in
$\pi(d)$, up to a small technical detail, becomes $\equiv 0\pmod{m}$,
violating ($\ast$ ‣ 2)) with each occurrence. Here, we use the notation $()$
to mean the empty composition, which has no parts and size 0.

1: Initialize with $i=\ell(c)$, $j=1$, and $d=()$

2: while $i\neq 0$ do

3: Let $j=j+c_{i}-1$.

4: if $c_{i}\not\equiv 0\;(\mathrm{mod}\;m)$ then

5: Append $c_{i}$ to the start of $d$ ($c_{i}$ becomes the new first part of
$d$)

6: else

7: Let $r$ be the remainder after dividing $j$ by $m$

8: Append the part $m-r$ to the start of $d$

9: Append the part $c_{i}-(m-r)$ to the start of $d$

10: end if

11: Let $i=i-1$

12: end while

From this construction, that $\lvert c\rvert=\lvert d\rvert$ is immediate.
Moreover, from ($\ast$ ‣ 2)), $j\not\equiv 0\;(\mathrm{mod}\;m)$ will hold at
every step, so in the case of lines 7 through 9, $r$ will never be 0, which
implies that the parts $m-r$ and $c_{i}-(m-r)$ are $\not\equiv
0\;(\mathrm{mod}\;m)$ as well. Therefore each part of $d$ is $\not\equiv
0\;(\mathrm{mod}\;m)$.

$\varphi(c)$ Figure 6. With $m=3$, $\varphi$ sends $c=(6,2,4,3,2)$ to
$d=(4,2,2,4,1,2,2)$.

Suppose that some composition $d$ is in the image of $\varphi$. Like in the
algorithm, we iterate $i$ from $d$’s last part to the top, with the goal of
uniquely determining the parts of the preimage. Start with $c=()$ and $j=1$,
and increment $j$ by $d_{i}-1$ at each step.

If $j\not\equiv 0\;(\mathrm{mod}\;m)$ then we must be in the case of line $5$,
so append $d_{i}$ to the top of $c$. Otherwise, we are in the case of lines 7
through 9 and so we must append $d_{i}+d_{i-1}$ to the start of $c$. We must
then subtract 1 from $i$ an additional time, and also add 1 to $j$ since the
action of splitting the composition part into two in $c$ has the effect of
decreasing $j$ by 1 in $d$. Continue until all parts of $d$ are exhausted,
which leaves a uniquely determined $c$. This shows injectivity which completes
the proof.

∎

###  3.1. Closing Remarks

If $n>m>2$ in Theorem 1.4 the inequality becomes strict since the compositions
of size $n$

$(m-1,2,\underbrace{1,\dots,1}_{n-m-1\;\mathrm{many\;1^{\prime}s}})$

cannot be in the image of $\varphi$. This fact can also be obtained from the
original proof. If $n\leq m$ both families of partitions coincide giving an
equality. Also, if $m=2$, $\varphi$ becomes the identity map, which provides
another proof of Straub’s theorem.

Many papers that treat compositions have noted that the following families of
compositions are all enumerated by the $n$th Fibonacci number $F_{n}$. Recall
that $F_{0}=0,F_{1}=1$, and $F_{n}=F_{n-1}+F_{n-2}$ for any $n\geq 2$.

  * 1)

Compositions of $n$ with odd parts.

  * 2)

Compositions of $n$ with parts 1 and 2, and last part 1.

  * 3)

Compositions of $n+1$ with parts greater than 1.

Lemma 3.1 with $m=2$ and $R=\\{1\\}$ gives a bijection from 1) to 2), and by
identifying 3) with compositions of $n$ where only the last part may be 1,
Proposition 2.3 with $m=2$ gives a bijection from 2) to 3).

## References

  * [1] T. Amdeberhan, Andrews G. E., and C. Ballantine, _Refinements of beck-type partition identities_ , Discrete Mathematics 345 (2022), no. 12.
  * [2] S. Fu. and D. Tang, _Partitions with fixed largest hook length_ , Ramanujan Journal (2018), 375–390.
  * [3] J. Huang, _Compositions with restricted parts_ , Discrete Mathematics 343 (2020), no. 7, 111875.
  * [4] Z. Lin, H. Xiong, and S. Yan, _Combinatorics of integer partitions with prescribed perimeter_ , Journal of Combinatorial Theory, Series A 197 (2023), 105747.
  * [5] A. Munagi, _Euler-type identities for integer compositions via zig-zag graphs_ , integers 12 (2012), no. A60.
  * [6] A. Munagi, _Integer compositions and higher-order conjugation_ , Journal of Integer Sequences 21 (2018), 1–22.
  * [7] A. Straub, _Core partitions into distinct parts and an analog of Euler’s theorem_ , European Journal of Combinatorics 57 (2016), 40–49.