# Improving Uncertainty Calibration via Prior Augmented Data

Jeffrey Willette  Juho Lee  Sung Ju Hwang

###### Abstract

Neural networks have proven successful at learning from complex data
distributions by acting as universal function approximators. However, they are
often overconfident in their predictions, which leads to inaccurate and
miscalibrated probabilistic predictions. The problem of overconfidence becomes
especially apparent in cases where the test-time data distribution differs
from that which was seen during training. We propose a solution to this
problem by seeking out regions of feature space where the model is
unjustifiably overconfident, and conditionally raising the entropy of those
predictions towards that of the prior distribution of the labels. Our method
results in a better calibrated network and is agnostic to the underlying model
structure, so it can be applied to any neural network which produces a
probability density as an output. We demonstrate the effectiveness of our
method and validate its performance on both classification and regression
problems, applying it to recent probabilistic neural network models.

Machine Learning, ICML

Figure 1: Performance of different models on OOD test data. The top row
contains baseline models, the bottom row contains baseline + PAD. It can be
seen that PAD effectively increases the epistemic uncertainty in regions with
sparse training data and exhibits a more predictable reversion to the prior,
much like a GP (top right). Baseline BNN models in the top row show
unpredictable behavior in OOD regions of input space.

##  1 Introduction

Deep neural networks (DNN) have proven successful on diverse tasks due to
their ability to learn highly expressive task-specific representations.
However, they are known to be overconfident when presented with new and unseen
inputs from the target distribution. Probabilistic networks should be accurate
in terms of both accuracy and calibration. Accuracy measures how often the
model’s predictions agree with the labels in the dataset, while Calibration
measures the precision of the uncertainty around a probabilistic output. For
example, an event predicted with 10% probability should be the empirical
outcome 10% of the time. The probability around rare but important outlier
events needs to be trustworthy for mission critical tasks such as autonomous
driving, medical diagnosis, and robotics applications.

Bayesian neural networks (BNN) and ensembling methods are popular and
effective ways to obtain uncertainty around a prediction for DNNs. Since Gal &
Ghahramani (2015) showed that Monte Carlo Dropout acts as a Bayesian
approximation, there have been numerous advances in modeling predictive
uncertainty with BNNs. As laid out by Kendall & Gal (2017), models need to
account for sources of both aleatoric and epistemic uncertainty. Epistemic
uncertainties arise from uncertainty in knowledge or ”knowledge gaps” for
which there may be novel inputs and outcomes. For BNNs, this presents as
uncertainty in the parameters which are trained to encode learned knowledge
about a data distribution. Aleatoric uncertainties arise from irreducible
noise in the data. Correctly modeling both forms of uncertainty is essential
in order to form accurate and calibrated predictions.

Accuracy and calibration are negatively impacted when data seen during
inference varies substantially from that which was seen during training. It
has been shown that when test data has undergone a significant distributional
shift from the training data, one can witness performance degradation across a
variety of DNN models (Snoek et al., 2019). A recurring result from Snoek et
al. (2019) is that Deep Ensembles (Lakshminarayanan et al., 2017) show
superior performance on out of distribution (OOD) test data. Previous work has
also shown that DNN’s fail to accurately model epistemic uncertainty, as
regions with sparse amounts of training data often lead to confident
predictions even when evidence to justify such confidence is lacking (Sun et
al., 2019). Bayesian non-parametric models such as Gaussian processes (GP)
also model epistemic uncertainties, but suffer from limited expressiveness and
the need to specify a kernel a priori which be infeasible for distributions
with unknown structure. In this work, we propose a new method for achieving
calibrated models by providing generated samples from a variational
distribution which augments the natural data to seek out areas of feature
space for which the model exhibits unjustifiably low levels of epistemic
uncertainty. For those regions of feature space, the model is encouraged to
predict uncertainty closer to that of the label prior. Our method can be
applied to any existing neural network model during training, in an arbitrary
feature space, and results in improved likelihood and calibration on OOD test
data.

Our contributions in this work are as follows:

  * •

We propose a new method of data augmentation, which we dub Prior Augmented
Data (PAD) that seeks to generate samples in areas where the model has an
unjustifiably low level of epistemic uncertainty.

  * •

We introduce a method for creating distributionally shifted OOD test sets for
regression problems, akin to the OOD test set image corruptions proposed by
Hendrycks & Dietterich (2019), which to the best of our knowledge, has not
been applied to regression calibration testing.

  * •

We experimentally validate our method on shifted data distributions for both
regression and classification tasks, on which it significantly improves both
the likelihood and calibration of a number of state-of-the-art probabilistic
neural network models.

##  2 Background

For regression tasks, we denote a set of features
$\mathbf{x}\in\mathbb{R}^{d}$ and labels $y\in\mathbb{R}$ which make up a
dataset of i.i.d. samples
$\mathcal{D}=\\{(\mathbf{x}_{n},y_{n})\\}_{n=1}^{N}$, with
$\mathbf{X}:=\\{\mathbf{x}_{n}\\}_{n=1}^{N}$. Let $f_{\theta}$ be a neural
network which is parameterized by weights $\theta$. Let the output of
$f_{\theta}(\mathbf{x})$ be a probability density $p_{\theta}(y|\mathbf{x})$
which is either in the form of a Gaussian density $\mathcal{N}(\mu,\sigma)$
for single task regression or a categorical distribution in the case of multi-
class classification. Let $g_{\phi}$ be a generative model which generates
pseudo inputs for $f_{\theta}$. We assume that both $f_{\theta}$ and
$g_{\phi}$ are iteratively trained via mini-batch stochastic gradient descent
with updates to generic model parameters $\tau$ given by the update rule
$\tau_{t+1}=\tau_{t}-\nabla_{\tau_{t}}\mathcal{L}$, with $\mathcal{L}$
representing a loss function which is differentiable w.r.t generic parameters
$\tau$. We refer to an out-of-distribution (OOD) or distributionally shifted
dataset $\tilde{\mathcal{D}}$ as one which is drawn from a different region of
the distribution than the training dataset $\mathcal{D}$. This distributional
shift can occur naturally for multiple reasons including a temporal modal
shift, or an imbalance in training data which may come about when gathering
data is more difficult or costly in particular regions of $\mathcal{D}$.

###  2.1 Bayesian Neural Networks

BNNs are neural networks with a distribution over the weights which aim to
model epistemic uncertainty in the weight space. In practice, this is often
done by introducing a variational distribution and then minimizing the
Kullback-Leibler divergence (Kullback, 1997) between the variational
distribution and the true weight posterior. For a further discussion of this
topic, we refer the reader to existing works (Kingma & Welling, 2013; Blundell
et al., 2015; Gal & Ghahramani, 2015). During inference, BNNs make predictions
by approximating the following integral with Monte Carlo samples from the
variational distribution $q(\theta|\mathcal{D})$.

$\vspace{0.5cm}\begin{split}p(y|\mathbf{x},\mathcal{D})&=\int
p_{\theta}(y|\mathbf{x})p(\theta|\mathcal{D})d\mathbf{\theta}\\\
&\approx\frac{1}{S}\sum_{s=1}^{S}p_{\theta_{s}}(y|\mathbf{x})q(\theta_{s}|\mathcal{D}),\\\
&\theta_{1},\dots,\theta_{S}\overset{\text{i.i.d.}}{\sim}q(\theta|\mathcal{D}).\end{split}$
(1)

###  2.2 Misplaced Confidence

A problem arises when neural networks do not accurately model the true
posterior over the weights $p(\mathbf{\theta}|\mathcal{D})$ given in (2). Our
conjecture is that a major factor contributing to generalization error in both
likelihood and calibration is a failure to revert to the prior $p(\theta)$ for
regions of the input space with insufficient evidence to warrant low entropy
predictions. Bayesian non-parametric models such as Gaussian processes (GP)
solve this through utilizing a kernel which makes pairwise comparisons between
all datapoints. GP’s come with the drawback of having to specify a kernel a
priori and are generally outperformed by DNNs which are known to be more
expressive than a GP with a common RBF kernel.

$p(\theta|\mathcal{D})=\frac{p(\mathcal{D}|\theta)p(\theta)}{\int
p(\mathcal{D}|\theta^{\prime})p(\theta^{\prime})d\theta^{\prime}}$ (2)

The problem of misplaced confidence of neural network was first studied by Guo
et al. (2017) which showed that modern DNNs exhibit poor correlation between
accuracy and confidence, which is known as expected calibration error (ECE),
and are often overconfident rather than underconfident. Snoek et al. (2019)
came to the general conclusion that Deep Ensembles (Lakshminarayanan et al.,
2017) tend to be the best calibrated model on OOD test data. Since then, there
have been a number of newly proposed BNN models such as SWAG (Snoek et al.,
2019) Multi-SWAG (Wilson & Izmailov, 2020), and Rank One Bayesian Neural
Networks (R1BNN) (Dusenberry et al., 2020), each of which utilize different
strategies for modeling the epistemic uncertainty in $p(\theta|D)$.

To illustrate the problem of failing to revert to the prior in underspecified
regions of the training distribution, we have provided a toy example in
(figure 1). The true function is given by
$x+\epsilon+\sin(4(x+\epsilon))+\sin(13(x+\epsilon))$, where
$\epsilon\sim\mathcal{N}(0,0.03)$. We then sample 100 points in the range of
$[0,0.4]$ and 100 points in the range of $[0.8,1.0]$, which leaves a gap in
the training data. One can observe in (Figure 1) how different neural network
models tend to make confident predictions in regions where they have not
observed any data, and exhibit unpredictable behavior around the outer
boundaries of the dataset. Our method encourages a reversion to the prior in
uncertain regions by generating pseudo OOD data and encouraging higher entropy
prediction on those data. In addition to covering the gap between the
datapoints, our model exhibits more predictable behavior around the outer
boundaries of the data when compared to other BNN models.

##  3 Method

To encourage a reversion towards the prior in uncertain regions, we learn to
generate pseudo OOD data which leads to a better calibrated model by raising
the entropy of the OOD predictions. An important design choice that we make is
for our OOD generator network $g_{\phi}(\cdot)$ to take a _dataset_
$\mathbf{X}$ as input to produce distributions of OOD data. The goal of the
OOD generator is to fill the “gaps” between the training data, using all
available current knowlege — the training data itself.

Once trained, the model $f_{\theta}$ should predict more uncertainty for data
generated from $g_{\phi}$. Raising the entropy of uninformative noisy psuedo
inputs may be a solution, but also could be too distant from the natural data,
and therefore provide no useful information for $f_{\theta}$ to learn from.
Ideally, we want realistic OOD data that are not too distant from the training
data and still predicted with more uncertainty by $f_{\theta}$. To achieve
this, we employ an adversarial training procedure similar to generative
adversarial nets (Goodfellow et al., 2014a) — we train $g_{\phi}$ to find
where $f_{\theta}$ may be overconfident, and at the same time train
$f_{\theta}$ to defend against this by predicting higher levels of uncertainty
in those regions. In the next section, we explain the objectives of
$g_{\phi}$.

###  3.1 The OOD sample generator network $g_{\phi}$

$g_{\phi}$ takes a dataset $\mathbf{X}=\\{\mathbf{x}_{i}\\}_{i=1}^{n}$ and
produces a distribution of an equally sized pseudo dataset
$\tilde{\mathbf{X}}=\\{\tilde{\mathbf{x}}_{n}\\}_{n=1}^{N}$. First, each
$\mathbf{x}_{i}$ is encoded via a feedforward network $g_{\text{enc}}(\cdot)$
to construct a set of representations
$\mathbf{Z}=\\{\mathbf{z}_{n}=g_{\text{enc}}(\mathbf{x}_{n})\\}_{n=1}^{N}$.
Then, we pick a subset size $K\in[1,\lfloor N/2\rfloor]$ for each mini-batch,
and for each $n=1,\dots,N$,

Figure 2: Left: A random train/test split. The test data is randomly chosen
from the whole distribution. Middle: Clusters resulting from spectral
clustering. Right: Shifted data distributions chosen from clusters as a
train/test split for our regression experiments on OOD data. (full method
outlined in section 4.1)

$\displaystyle g_{\phi}(\mathbf{X})_{n}$
$\displaystyle=g_{\text{dec}}\bigg{(}\frac{1}{K}\sum_{m\in\text{nn}_{K}(n)}\mathbf{z}_{m}\bigg{)},$
(3)

where $\text{nn}_{K}(m)$ denotes the set of $K$-nearest neighbors of
$\mathbf{z}_{n}$ and $g_{\text{dec}}$ is another feedforward network. The
distribution for the generated data $\tilde{\mathbf{X}}$ is then defined as

$\displaystyle
q_{\phi}(\tilde{\mathbf{X}}|\mathbf{X})=\prod_{n=1}^{N}q_{\phi}(\tilde{\mathbf{x}}_{n}|g_{\phi}(\mathbf{X})_{n}).$
(4)

As stated previously, our goal is to generate psuedo-OOD data, but we cannot
be sure where such data will arise from. The only thing we can be certain of
is that 1) the training data exist, and 2) there exists some distribution
which is OOD to the training data. Therefore, our only option is to learn
directly from representations of training data in order to find likely regions
of OOD data in ’knowledge gaps’.

###  3.2 Training objectives

#### Training $g_{\phi}$

Given a batch of data $\mathbf{X}_{B}=\\{\mathbf{x}_{n}\\}_{n=1}^{B}$, we
first construct the OOD distribution
$g_{\phi}(\tilde{\mathbf{X}}_{B}|\mathbf{X}_{B})$ via $g_{\phi}$. The training
loss for $\mathbf{X}_{B}$ is defined as

$\footnotesize\begin{split}\ell_{\phi}(\mathbf{X}_{B})&=\frac{1}{B}\sum_{n=1}^{N}\Big{(}\underbrace{\mathbb{E}_{q_{\phi}(\tilde{\mathbf{x}}_{n})}\Big{[}\mathcal{H}[p_{\theta}(y|\tilde{\mathbf{x}}_{n})]\Big{]}}_{\text{A}}-\underbrace{\mathcal{H}[q_{\phi}(\tilde{\mathbf{x}}_{n})]}_{\text{B}}\Big{)}\\\
&+\underbrace{\frac{1}{BK}\sum_{n=1}^{B}\sum_{m\in\text{nn}_{K}(n)}\mathbb{E}_{q_{\phi}(\tilde{\mathbf{x}}_{n})}[\|\tilde{\mathbf{x}}_{n}-\mathbf{x}_{m}\|^{2}]}_{\text{C}},\end{split}$
(5)

where
$q_{\phi}(\tilde{\mathbf{x}}_{n}):=q_{\phi}(\tilde{\mathbf{x}}_{n}|g_{\phi}(\mathbf{X}_{B})_{n})$
and $\mathcal{H}[p]:=-\int p(\mathbf{x})\log p(\mathbf{x})d\mathbf{x}$ is the
entropy, and $\text{nn}_{K}(m)$ in C is the set of $K$-nearest neighbors of
$\tilde{\mathbf{x}}_{n}$ among $\mathbf{X}_{B}$. The A term trains $g_{\phi}$
to fool $f_{\theta}$ by seeking regions where
$f_{\theta}(\tilde{\mathbf{x}}_{n})$ makes low entropy predictions, i.e.,
where it may be overconfident on OOD data. The B term encourages diversity of
the generated samples by maximizing the entropy of
$q_{\phi}(\tilde{\mathbf{x}}_{n})$. Without B, the generated samples would be
prone to mode collapse and become homogeneous and uninformative. The final
term, C, minimizes the average pairwise distance between the real data and the
generated data it is conditioned on, where $\mathbf{x}_{m}\in\mathbf{X}_{B}$.
As the C term minimizes this distance, the generated data is likely to exist
in the ”gap” regions of the natural training data. Combined with the training
loss for $f_{\theta}$ which we will describe next, we aim to train $g_{\phi}$
to produce $\tilde{\mathbf{x}}$ that are not too distant from the training
data but still differ enough to warrant an increase in uncertainty. We provide
an ablation study to see the effects of the terms A, B, and C, in tables 4and
4.

Given the training data $\mathcal{D}$, we optimize the expected loss
$\mathcal{L}_{\phi}:=\mathbb{E}_{\mathbf{X}_{B}}[\ell_{\phi}(\mathbf{X}_{B})]$
over subsets of size $B$ obtained from $\mathbf{X}$. In practice, at each
step, we sample a single mini-batch $\mathbf{X}_{B}$ to compute the gradient.
Also, we choose $q_{\phi}(\tilde{\mathbf{x}}_{n})$ to be a reparameterizable
distribution, and approximate the expectation over $\tilde{\mathbf{x}}_{n}$
via a single sample $\tilde{\mathbf{x}}_{n}\sim
q_{\phi}(\tilde{\mathbf{x}}_{n})$.

#### Training $f_{\theta}$

For $f_{\theta}$, we minimize the expected loss,

$\displaystyle\mathcal{L}_{\theta}=\mathbb{E}_{(\mathbf{x},y)}[-\log
p_{\theta}(y|\mathbf{x})]+\mathbb{E}_{\mathbf{X}_{B}}[r_{\theta}(\mathbf{X}_{B})],$
(6)

where the first term is the negative log-likelihood of the natural data, and
the second term is a regularizer which encourages more uncertainty on OOD
data,

$\begin{split}r_{\theta}(\mathbf{X}_{B}):=\frac{1}{B}\sum_{n=1}^{B}\mathbb{E}_{q_{\phi}(\tilde{\mathbf{x}}_{n})}\left[\lambda_{n}\mathbb{KL}[p_{\theta}(y|\tilde{\mathbf{x}}_{n})\|p(y)]\right],\end{split}$
(7)

where $\lambda_{n}$ is a weighting term with $\ell>0$ as a scaling
hyperparameter. which approaches 1 as $\mathbf{\tilde{x}}$ becomes further
from the natural data distribution, and approaches 0 as
$\mathbf{\tilde{x}}\rightarrow\mathbf{x}$.

$\lambda_{n}:=1-\exp{\Big{(}-\frac{\min{||\mathbf{\tilde{x}}_{n}-\mathbf{x}_{m}||^{2}}}{2\ell^{2}}\Big{)}}$
(8)

The role of the regularizer is to encourage predictions on the OOD data to be
closer to the prior $p(y)$. Note that the KL term decays towards zero as
$\mathbf{x}$ becomes closer to $\tilde{\mathbf{x}}$. This allows $f_{\theta}$
the freedom to make confident predictions in regions where real data exist. We
draw a mini-batch $\mathbf{X}_{B}$ from $\mathbf{X}$, approximating
$r_{\theta}(\mathbf{X}_{B})$ with a single sample $\tilde{\mathbf{X}}_{B}\sim
q_{\phi}(\tilde{\mathbf{X}}_{B}|\mathbf{X}_{B})$. The loss is then
approximated by (9),

$\mathcal{L}_{\theta}\approx-\frac{1}{B}\sum_{n=1}^{B}\log
p_{\theta}(y_{n}|\mathbf{x}_{n})+r_{\theta}(\mathbf{X}_{B})$ (9)

In terms of added complexity, PAD optimizes one additional set of parameters
$\phi$ and requires an alternating pattern of training akin to that of GAN’s
(Goodfellow et al., 2014a). In our experiments, we utilize a shallow network
with a single hidden layers for $\phi$. For further information about how we
implement the $\mathbb{KL}$ term in (6) we we refer the reader to section 5
for implementation details.

##  4 Experiments

Table 1: Negative log likelihood on UCI regression datasets. GP’s and FVBNN’s
are included for reference. bold entries contain the best result for the base
model. underlined entries are those with a large difference of $\geq 1$ (log
scale) between methods.

Model | Housing | Concrete | Energy | Kin8nm | Naval | Power | Wine | Yacht
---|---|---|---|---|---|---|---|---
GP |  3.81$\pm$0.23 |  4.44$\pm$0.08 |  6.74$\pm$7.44 |  -0.50$\pm$0.04 |  -3.55$\pm$2.07 |  4.14$\pm$0.36 |  1.22$\pm$0.05 |  4.31$\pm$0.21
FVBNN |  4.62$\pm$1.08 |  4.80$\pm$0.50 |  3.16$\pm$1.02 |  0.01$\pm$0.26 |  -3.19$\pm$0.40 |  3.18$\pm$0.10 |  1.30$\pm$0.00 |  3.26$\pm$0.00
DUN |  4.86$\pm$1.64 |  4.78$\pm$0.73 |  4.41$\pm$1.22 |  0.85$\pm$1.49 |  -1.16$\pm$0.74 |  4.34$\pm$1.22 |  3.61$\pm$7.06 |  6.37$\pm$3.10
DE |  6.83$\pm$2.92 |  8.34$\pm$4.66 |  10.73$\pm$23.25 | -1.03$\pm$0.23 |  -0.42$\pm$1.92 |  3.12$\pm$0.18 |  2.18$\pm$0.67 | 2.39$\pm$0.64
DE PAD | 3.61$\pm$0.22 | 5.13$\pm$0.98 | 3.24$\pm$0.62 |  -0.38$\pm$0.10 | -2.81$\pm$0.36 | 3.12$\pm$0.08 | 1.24$\pm$0.08 |  3.78$\pm$0.07
R1BNN |  4.18$\pm$0.70 |  5.17$\pm$1.27 |  9.22$\pm$19.74 | -0.86$\pm$0.19 |  3.83$\pm$10.40 | 2.95$\pm$0.12 |  1.72$\pm$1.34 | 3.29$\pm$0.64
R1BNN PAD | 3.84$\pm$0.21 | 4.30$\pm$0.18 | 3.78$\pm$0.18 |  0.09$\pm$0.22 | -3.25$\pm$0.20 |  3.13$\pm$0.05 | 1.27$\pm$0.10 |  4.14$\pm$0.03
SWAG |  5.02$\pm$2.26 |  4.85$\pm$1.00 |  3.73$\pm$2.83 | -0.96$\pm$0.18 | -2.96$\pm$0.37 | 3.03$\pm$0.14 |  1.41$\pm$0.39 |  4.25$\pm$0.71
SWAG PAD | 3.80$\pm$0.58 | 4.47$\pm$0.29 | 3.51$\pm$1.69 |  -0.58$\pm$0.09 |  -2.46$\pm$0.48 |  3.18$\pm$0.12 | 1.20$\pm$0.09 | 3.53$\pm$0.10
MC Drop |  5.37$\pm$1.42 |  5.88$\pm$1.79 |  4.04$\pm$4.23 | -0.82$\pm$0.23 |  4.93$\pm$7.55 | 3.09$\pm$0.15 |  1.70$\pm$0.53 | 3.21$\pm$2.05
MC Drop PAD | 4.32$\pm$1.92 | 4.98$\pm$0.79 | 3.34$\pm$0.86 |  -0.53$\pm$0.09 | -0.96$\pm$3.39 |  3.18$\pm$0.08 | 1.22$\pm$0.07 |  3.45$\pm$0.14

###  4.1 Experimental Setup and Datasets

In order to create a shifted OOD test set for regression problems, we first
run a spectral clustering algorithm to get 10 separate clusters on each
dataset. We then randomly choose test clusters until we have a test set which
is $\geq 20\%$ of the total dataset size, using the remaining clusters for
training. We repeat this process 10 times for each dataset. A TSNE
visualization of a dataset created this way is given in figure 2. We do this
to ensure there is a significant shift between training and testing data.

For regression, we consider UCI datasets (Dua & Graff, 2017) following
Hernández-Lobato & Adams (2015). The base network is a multi-layer perceptron
with two hidden layers of 50 units with ReLU activations. The generator uses a
single hidden layer of 50 units for both the encoder and decoder and uses a
permutation invariant pooling layer consisting of $[mean(x),max(x)]$. The
encoding of $\mathbf{X}$ and generation of $\mathbf{\tilde{X}}$ are done
directly in the input space. We train the models for a total of 50 epochs on
each of the 10 dataset splits. We report both negative log likelihood (NLL)
and calibration error for regression as proposed by Kuleshov et al. (2018)
measured with 100 bins on the cumulative distribution function (CDF) of the
density $p_{\theta}(y_{i}|\mathbf{x}_{i})$. For baseline models, we tune
hyperparameters with a randomly chosen 80/20 training/validation split from
the training set. For PAD models, we do k-fold cross validation with $K=2$,
selecting half of the previously made clusters for each fold. For all models
and datasets, we tune the hyperparameters for each individual train/test
split.

Classification experiments are done on both MNIST (LeCun et al., 2010) and
CIFAR-10 (Krizhevsky et al., 2009) datasets. We use an architecture consisting
of 4 and 5 convolutional layers respectively, followed by 3 fully connected
layers. We provide extra information regarding the exact architecture in the
supplementary file. Instead of clustering to create shifted test distributions
as was done in the regression experiments, we use image corruptions as were
used by Hendrycks & Dietterich (2019); Snoek et al. (2019) for OOD test data.
As PAD works in any arbitrary feature space, we apply our method in the latent
space between the last convolutional layer and the fully connected classifier.
We report classification accuracy, negative log likelihood, and expected
calibration error (ECE) (Guo et al., 2017) over 5 runs for all models.

Table 2: Calibration error on UCI regression datasets. GP’s and FVBNN’s are
included for reference. bold entries contain the best result for the base
model. underlined entries are those with a large difference of $\geq 5$
between methods.

Model | Housing | Concrete | Energy | Kin8nm | Naval | Power | Wine | Yacht
---|---|---|---|---|---|---|---|---
GP |  8.24$\pm$2.73 |  3.34$\pm$2.68 |  4.88$\pm$3.14 |  2.17$\pm$0.95 |  6.65$\pm$7.39 |  4.30$\pm$4.72 |  1.52$\pm$1.07 |  5.10$\pm$3.50
FVBNN |  7.10$\pm$7.76 |  7.79$\pm$5.77 |  5.90$\pm$8.62 |  4.46$\pm$5.06 |  3.23$\pm$2.41 |  2.06$\pm$1.47 |  2.10$\pm$0.00 |  2.46$\pm$0.00
DUN |  15.8$\pm$21.5 |  14.3$\pm$10.9 |  19.5$\pm$23.9 |  16.5$\pm$14.0 |  22.4$\pm$5.4 |  19.3$\pm$20.7 |  18.7$\pm$24.4 |  18.9$\pm$16.9
DE |  15.95$\pm$6.97 |  18.83$\pm$8.66 | 5.54$\pm$8.91 | 1.44$\pm$1.36 |  8.06$\pm$5.77 |  2.56$\pm$2.53 |  2.81$\pm$1.14 |  6.61$\pm$6.12
DE PAD | 5.12$\pm$3.62 | 12.87$\pm$6.66 |  6.34$\pm$7.08 |  2.38$\pm$1.89 | 6.19$\pm$3.93 | 1.60$\pm$1.48 | 1.53$\pm$1.05 | 2.37$\pm$0.18
R1BNN |  6.42$\pm$5.45 |  14.18$\pm$8.77 |  4.82$\pm$9.37 | 1.63$\pm$1.57 |  6.41$\pm$4.93 | 1.07$\pm$1.25 |  2.29$\pm$1.50 |  2.82$\pm$0.61
R1BNN PAD | 6.04$\pm$3.93 | 2.94$\pm$3.10 | 4.42$\pm$3.83 |  4.63$\pm$5.06 | 3.77$\pm$2.75 |  1.38$\pm$0.61 | 2.14$\pm$1.39 | 2.18$\pm$0.09
SWAG |  12.43$\pm$6.84 |  10.16$\pm$6.79 | 5.74$\pm$8.85 | 1.62$\pm$1.78 | 2.98$\pm$2.36 | 1.57$\pm$2.19 |  1.85$\pm$2.26 |  4.91$\pm$1.55
SWAG PAD | 6.73$\pm$6.44 | 8.72$\pm$4.93 |  6.05$\pm$8.60 |  2.15$\pm$0.77 |  4.59$\pm$1.69 |  1.91$\pm$1.87 | 0.74$\pm$0.51 | 2.76$\pm$0.45
MC Drop |  11.48$\pm$6.17 |  14.92$\pm$9.90 | 5.98$\pm$8.94 | 2.26$\pm$2.31 |  5.75$\pm$2.88 |  2.33$\pm$2.17 |  1.46$\pm$0.73 |  6.74$\pm$6.95
MC Drop PAD | 8.38$\pm$8.32 | 12.78$\pm$6.71 |  6.30$\pm$7.71 |  2.30$\pm$1.29 | 5.52$\pm$3.96 | 1.90$\pm$1.66 | 1.39$\pm$1.11 | 3.58$\pm$0.81

###  4.2 Baselines

We compare PAD against a number of Bayesian models and neural networks
including Gaussian Processes, Functional Variational Bayesian Neural Networks
(FVBNN) (Sun et al., 2019), Monte Carlo Dropout (MC Drop) (Gal & Ghahramani,
2015), Deep Ensembles (DE) (Lakshminarayanan et al., 2017), SWAG (Maddox et
al., 2019), and Rank One Bayesian Neural Networks (R1BNN) (Dusenberry et al.,
2020) and Depth Uncertain Networks (DUN) (Antorán et al., 2020).

###  4.3 Analysis

For regression it can be seen in table 1 that PAD generally improves the
likelihood in the scenario where the shifted data distribution caused the
baseline model to perform poorest. The underlined entries in (table 1) are
those which the negative log likelihood differs by $\geq 1$ on the log scale.
It can be seen that the majority (8/9) of these increases in likelihood are
achieved by PAD models. In terms of single experiments, PAD beat the baseline
models $21/32\approx 66\%$ of the time. For calibration, in (table 2), the
only difference is that the underlined entries are those for which differ by
$\geq 5$. For calibration, it can be seen that PAD models contain all cases
where a large difference between methods is present. In terms of single
experiments, PAD beat the baselines $22/32\approx 69\%$ of the time.

###  4.4 Ablation Study

In order to understand the effect of each term in equation 5, we performed an
ablation on each term using the MC PAD (MC Dropout + PAD) variant. It can be
seen that as more terms are removed from the equation, the model performance
degrades in both NLL and calibration. The effect is more pronounced for NLL,
as the full equation contains the majority of the best performances. There are
a nontrivial amount of the best performances which occurred without the A term
added to the loss, but most of the bold entries in the ”Without A” column are
very close to the values achieved by the full loss function. Therefore, the A
term may be considered optional, dependent on the given task, but still has an
important theoretical justification in that it seeks out the overconfident
areas of $f_{\theta}$.

Table 3: Equation 5 Ablation: negative log likelihood when progressively
removing terms A, B, and C.

Dataset | Regular | Without A | Without B | Without AB
---|---|---|---|---
Housing | 4.32$\pm$1.92 |  4.62$\pm$2.31 |  4.69$\pm$2.30 |  4.70$\pm$2.31
Concrete |  4.98$\pm$0.79 | 4.96$\pm$0.73 |  5.05$\pm$0.74 |  5.05$\pm$0.74
Energy |  3.34$\pm$0.86 | 3.26$\pm$1.00 |  3.34$\pm$1.39 |  3.34$\pm$1.39
Kin8nm | -0.53$\pm$0.09 |  -0.53$\pm$0.09 |  -0.48$\pm$0.11 |  -0.48$\pm$0.11
Naval | -0.96$\pm$3.39 |  0.21$\pm$3.77 |  0.31$\pm$3.69 |  0.22$\pm$3.20
Power | 3.18$\pm$0.08 |  3.23$\pm$0.10 |  3.23$\pm$0.20 |  3.23$\pm$0.20
Wine | 1.22$\pm$0.07 |  1.34$\pm$0.57 |  1.72$\pm$1.71 |  1.70$\pm$1.66
Yacht |  3.45$\pm$0.14 |  3.45$\pm$0.11 |  3.41$\pm$0.13 | 3.41$\pm$0.13

Table 4: Equation 5 Ablation: calibration error when progressively removing
terms A, B, and C.

Dataset | Regular | Without A | Without B | Without AB
---|---|---|---|---
Housing | 8.38$\pm$8.32 |  9.43$\pm$8.22 |  9.69$\pm$8.35 |  9.70$\pm$8.35
Concrete | 12.78$\pm$6.71 |  14.04$\pm$7.44 |  14.53$\pm$7.31 |  14.53$\pm$7.31
Energy | 6.30$\pm$7.71 |  6.40$\pm$7.89 |  6.53$\pm$8.29 |  6.52$\pm$8.30
Kin8nm |  2.30$\pm$1.29 | 2.24$\pm$1.43 |  2.47$\pm$1.61 |  2.47$\pm$1.61
Naval |  5.52$\pm$3.96 | 5.44$\pm$3.08 |  5.88$\pm$3.94 |  5.65$\pm$3.71
Power |  1.90$\pm$1.66 |  1.97$\pm$1.56 | 1.83$\pm$1.76 |  1.83$\pm$1.82
Wine |  1.39$\pm$1.11 |  0.50$\pm$0.31 |  0.50$\pm$0.31 | 0.48$\pm$0.28
Yacht |  3.58$\pm$0.81 | 3.42$\pm$0.74 |  3.44$\pm$0.82 |  3.44$\pm$0.82

###  4.5 New York Real Estate

To apply PAD on a real world problem where the dataset shift is not
synthetically created, we applied it to regression on New York real estate
data. The dataset consists of 12,000 sales records spanning over 12 years.
Each instance has a total of 667 features including real valued and one-hot
categorical features. We used the same base models as outlined in section 4.
The regression labels are the price that the house sold for in a given year.
We train the model to predict $\log(y)$ to account for the log-normal
distribution of prices and report results based on the log-transformed label.
We used the years of 2008-2009 as training/validation data and evaluated the
performance on all following years until 2019. It can be seen that with a
realistic temporal distributional shift, PAD models outperform the baseline in
both NLL and calibration error.

Figure 3: Log Likelihood and calibration error for a regression model trained
on one year of New York real estate data. PAD models maintain a better
likelihood and calibration error when given a natural temporally shifted
distribution.

Figure 4: MC Dropout (MC Drop) and Deep Ensemble (DE) performance on varying
degrees of shift intensity for MNIST-C and CIFAR10-C. 0 represents the
original test set while 5 represents the most extreme level of shift. Models
which are augmented with PAD show comparable performance on the natural test
set. As shift intensity increases, PAD models exhibit superior performance in
terms of negative log likelihood and calibration error.

##  5 Implementation Details

###  5.1 $\mathbb{KL}$ Divergences

For regression tasks we use an analytic calculation of
$\mathbb{KL}[p_{\theta}(y_{i}|\mathbf{x}_{i})||p(y)]$ by assuming a prior of
$\mathcal{N}(0,1)$. In practice, we only want to raise the uncertainty of the
prediction while keeping the expressive generalization properties of DNNs, so
we use the output of $p_{\theta}$ as the mean with the standard deviation set
to $1$ (equation 10). We also found that the constraint given by C in 5 was
sometimes too strong, given that the generation happens directly in the input
space and the datasets are generally of low dimensionality. For regression, we
therefore only started to penalize the C term distance when it exceeded a
certain boundary threshold. We used $||1^{d}||$, where $d$ is the
dimensionality of the inputs.

$\mathbb{KL}[p_{\theta}(y_{i}|\mathbf{x}_{i})||p(y)]:=\mathbb{KL}[\mathcal{N}_{\theta}(\mu_{i},\sigma_{i})||\mathcal{N}_{\theta}(\mu_{i},1)]$
(10)

For classification tasks, the label prior is a uniform categorical
distribution. In practice, we found that directly raising the entropy of the
output led to a degradation in accuracy, so we instead add another output
parameter to the base model such that instead of outputting $C$ class logits,
we output $C+1$ logits and treat the extra output logit as the log temperature
$\log t$ and use it as a temperature scaling parameter $\sigma(z/t)$ where
$\sigma$ is the softmax function. We then implement the $\mathbb{KL}$
divergence by setting a hyperparameter $\tau$ controlling the maximum
temperature, and conditionally raising the temperature by the following
function.

$T=(\tau^{\lambda}-t)^{2}$ (11)

where $w$ is the weight given by the exponential term before the $\mathbb{KL}$
divergence in the discriminative model loss. In this way, the minimum
temperature of $1$ is enforced when $\lambda=0$, and the maximum temperature
of $\tau$ is enforced when $\lambda=1$. Importantly, we do not transform the
class prediction logits when calculating the training classification loss
because we found that this led to a larger generalization error in practice.

Table 5: Classification accuracy, NLL, and calibration error on OOD test sets
for classification. Each row contains a baseline model, with Mixup and PAD
variants. Left: Accuracy, Middle: ECE, Right: NLL

ACCURACY
---
Model | MNIST | CIFAR-10
MC Drop |  0.61$\pm$0.01 | 0.45$\pm$0.01
MC Drop PAD | 0.62$\pm$0.01 |  0.45$\pm$0.01
DE | 0.68$\pm$0.00 | 0.52$\pm$0.00
DE PAD |  0.60$\pm$0.01 |  0.48$\pm$0.01
R1BNN |  0.69$\pm$0.03 |  0.43$\pm$0.01
R1BNN PAD | 0.69$\pm$0.04 | 0.45$\pm$0.00
SWAG |  0.68$\pm$0.01 | 0.59$\pm$0.00
SWAG PAD | 0.69$\pm$0.01 |  0.59$\pm$0.00
ECE
---
Model | MNIST | CIFAR-10
MC Drop |  0.31$\pm$0.02 |  0.38$\pm$0.02
MC Drop PAD | 0.07$\pm$0.07 | 0.05$\pm$0.01
DE |  0.24$\pm$0.01 |  0.26$\pm$0.01
DE PAD | 0.18$\pm$0.02 | 0.14$\pm$0.03
R1BNN |  0.10$\pm$0.05 | 0.06$\pm$0.02
R1BNN PAD | 0.07$\pm$0.04 |  0.07$\pm$0.04
SWAG |  0.22$\pm$0.01 | 0.10$\pm$0.00
SWAG PAD | 0.11$\pm$0.02 |  0.17$\pm$0.01
NLL
---
Model | MNIST | CIFAR-10
MC Drop |  2.36$\pm$0.26 |  3.23$\pm$0.40
MC Drop PAD | 1.20$\pm$0.11 | 1.69$\pm$0.02
DE |  1.52$\pm$0.11 |  2.06$\pm$0.03
DE PAD | 1.35$\pm$0.07 | 1.69$\pm$0.05
R1BNN |  1.47$\pm$0.58 |  1.76$\pm$0.01
R1BNN PAD | 0.98$\pm$0.07 | 1.67$\pm$0.01
SWAG |  1.75$\pm$0.18 | 1.28$\pm$0.01
SWAG PAD | 1.10$\pm$0.08 |  1.33$\pm$0.02

##  6 Related Work

#### Bayesian Methods for Deep Learning

Functional Bayesian Neural Networks (FVBNN) (Sun et al., 2019) use a
functional prior from a fully trained GP (Rasmussen, 2003). This approach is
somewhat similar to PAD, but PAD accomplishes this in a way which
adversarially generates difficult samples for $f_{\theta}$ and does not
require specifying a GP kernel and then fitting data to it with the limited
expessivity of a GP. (Maddox et al., 2019) recently proposed SWAG, which keeps
a moving average of the weights, and subsequently builds a multivariate
Gaussian posterior $p(\theta|\mathcal{D})$ which can be sampled for inference.
Deep ensembles (Lakshminarayanan et al., 2017) simply trains an ensemble of
$N$ independent models, and has become a strong standard baseline in the
calibration literature. The resulting ensemble of networks are sampled on new
inputs and combined in order to achieve set of highly probable solutions from
the posterior $p(\mathbf{w}|\mathcal{D})$. While simple and effective, it can
require excessive computation for large models and datasets. (Dusenberry et
al., 2020) proposed R1BNN, which uses a shared set of parameters $\theta$
along with multiple sets of rank one vectors $\mathbf{r}_{i}$ and
$\mathbf{s}_{i}$, which can combine via an outer product
$\theta\odot\mathbf{r}\mathbf{s}^{T}$ to create a parameter efficient
ensemble. Other recent advances include gathering ensemble members by treating
depth as a random variable and ensembling the latent features through a shared
output layer (Antorán et al., 2020).

#### Data Augmentation

The data augmentation literature is vast, and as such we will only mention
some of the most relevant works here. Adversarial examples (Goodfellow et al.,
2014b) utilize the gradient of the input with respect to the loss which is
then used to create a slightly perturbed input which would cause an increase
to the loss, and further training with that input. The resulting network
should be more robust to such perturbations. We note that our deep ensemble
(Lakshminarayanan et al., 2017) baseline uses this process in the training
procedure and is still outperformed by PAD. (Zhang et al., 2017) proposed
Mixup, which creates linear interpolations between natural inputs with the
hopes of creating a robust network with linear transitions between classes
instead of having a hard decision boundary. Subsequent work by (Thulasidasan
et al., 2019) showed that mixup training and the soft decision boundary has
the effect of improving network calibration.

#### Set Encoding

Methods which operate as a function of sets have been an active topic in
recent years. Deep Sets (Zaheer et al., 2017) first proposed a model
$f(\\{\mathbf{x}_{i},\mathbf{x}_{i+1},...,\mathbf{x}_{n}\\})\rightarrow\mathbb{R}^{d}$
which passes the input set through a feature extractor, before being
aggregated with a permutation invariant pooling function, and then decoded
through a DNN which projects the set representation to the output space
$\mathbb{R}^{d}$. (Lee et al., 2019) proposed to use attention layers to
create the set transformer which can model higher order interactions between
set members. Our method builds on the applications of set functions by using
the training set towards an objective of identifying and generating data in
underspecified regions where the model is likely overconfident.

##  7 Conclusion

We have proposed a new method of increasing the likelihood and calibration of
predictions from probabilistic neural network models which we named Prior
Augmented Data (PAD). Our method works by encouraging a Bayesian reversion to
the prior beliefs of the labels for inputs in previously unseen or sparse
regions of the known data distribution. We have demonstrated through extensive
experiments that our method achieves an improvement in likelihood and
calibration under numerous OOD data conditions, creating exponential
improvements in log likelihood in those conditions which the baseline models
tend to make the poorest predictions. One interesting direction of future work
could be to investigate a rigorous mathematical way of regularizing the weight
space to remain uncertain in unseen regions of the training data feature
space. If this were possible, our $g_{\phi}$ network could be replaced by a
simple addition of a regularization term to the loss function. Another
possible avenue of future research would be to look into the effect that
different sizes of $\mathbf{\tilde{X}}$ may have on the resulting uncertainty
predictions. It may be the case that the specific cluster sizes and manifold
geometries may require more/less attention and therefore some performance and
efficiency gains could be had by giving these regions the proper amount of
attention.

## References

  * Antorán et al. (2020) Antorán, J., Allingham, J., and Hernández-Lobato, J. M.  Depth uncertainty in neural networks.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Blundell et al. (2015) Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D.  Weight uncertainty in neural networks.  _arXiv preprint arXiv:1505.05424_ , 2015.
  * Dua & Graff (2017) Dua, D. and Graff, C.  UCI machine learning repository, 2017.  URL http://archive.ics.uci.edu/ml.
  * Dusenberry et al. (2020) Dusenberry, M. W., Jerfel, G., Wen, Y., Ma, Y.-a., Snoek, J., Heller, K., Lakshminarayanan, B., and Tran, D.  Efficient and scalable bayesian neural nets with rank-1 factors.  _arXiv preprint arXiv:2005.07186_ , 2020.
  * Gal & Ghahramani (2015) Gal, Y. and Ghahramani, Z.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  _arXiv preprint arXiv:1506.02142_ , 2015.
  * Goodfellow et al. (2014a) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.  Generative adversarial nets.  In _Advances in neural information processing systems_ , pp. 2672–2680, 2014a.
  * Goodfellow et al. (2014b) Goodfellow, I. J., Shlens, J., and Szegedy, C.  Explaining and harnessing adversarial examples, 2014b.
  * Guo et al. (2017) Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q.  On calibration of modern neural networks.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pp. 1321–1330. JMLR. org, 2017.
  * Hendrycks & Dietterich (2019) Hendrycks, D. and Dietterich, T.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hernández-Lobato & Adams (2015) Hernández-Lobato, J. M. and Adams, R.  Probabilistic backpropagation for scalable learning of bayesian neural networks.  In _International Conference on Machine Learning_ , pp. 1861–1869, 2015.
  * Kendall & Gal (2017) Kendall, A. and Gal, Y.  What uncertainties do we need in bayesian deep learning for computer vision?  In _Advances in neural information processing systems_ , pp. 5574–5584, 2017.
  * Kingma & Welling (2013) Kingma, D. P. and Welling, M.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G., et al.  Learning multiple layers of features from tiny images.  _https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf_ , 2009.
  * Kuleshov et al. (2018) Kuleshov, V., Fenner, N., and Ermon, S.  Accurate uncertainties for deep learning using calibrated regression.  _arXiv preprint arXiv:1807.00263_ , 2018.
  * Kullback (1997) Kullback, S.  _Information theory and statistics_.  Courier Corporation, 1997.
  * Lakshminarayanan et al. (2017) Lakshminarayanan, B., Pritzel, A., and Blundell, C.  Simple and scalable predictive uncertainty estimation using deep ensembles.  In _Advances in neural information processing systems_ , pp. 6402–6413, 2017.
  * LeCun et al. (2010) LeCun, Y., Cortes, C., and Burges, C.  Mnist handwritten digit database.  _ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist_ , 2, 2010.
  * Lee et al. (2019) Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., and Teh, Y. W.  Set transformer: A framework for attention-based permutation-invariant neural networks.  In _International Conference on Machine Learning_ , pp. 3744–3753, 2019.
  * Maddox et al. (2019) Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., and Wilson, A. G.  A simple baseline for bayesian uncertainty in deep learning.  In _Advances in Neural Information Processing Systems_ , pp. 13153–13164, 2019.
  * Rasmussen (2003) Rasmussen, C. E.  Gaussian processes in machine learning.  In _Summer School on Machine Learning_ , pp. 63–71. Springer, 2003\.
  * Snoek et al. (2019) Snoek, J., Ovadia, Y., Fertig, E., Lakshminarayanan, B., Nowozin, S., Sculley, D., Dillon, J., Ren, J., and Nado, Z.  Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift.  In _Advances in Neural Information Processing Systems_ , pp. 13969–13980, 2019.
  * Sun et al. (2019) Sun, S., Zhang, G., Shi, J., and Grosse, R.  Functional variational bayesian neural networks.  _arXiv preprint arXiv:1903.05779_ , 2019.
  * Thulasidasan et al. (2019) Thulasidasan, S., Chennupati, G., Bilmes, J. A., Bhattacharya, T., and Michalak, S.  On mixup training: Improved calibration and predictive uncertainty for deep neural networks.  In _Advances in Neural Information Processing Systems_ , pp. 13888–13899, 2019.
  * Wilson & Izmailov (2020) Wilson, A. G. and Izmailov, P.  Bayesian deep learning and a probabilistic perspective of generalization.  _arXiv preprint arXiv:2002.08791_ , 2020.
  * Zaheer et al. (2017) Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J.  Deep sets.  In _Advances in neural information processing systems_ , pp. 3391–3401, 2017.
  * Zhang et al. (2017) Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D.  mixup: Beyond empirical risk minimization.  _arXiv preprint arXiv:1710.09412_ , 2017.


# Schottky vertex operator cluster algebras

A. Zuevsky  Institute of Mathematics
Czech Academy of Sciences
Zitna 25, 11567
Prague
Czech Republic<EMAIL_ADDRESS>
###### Abstract.

Using recursion formulas for vertex operator algebra higher genus characters
with formal parameters identified with local coordinates around marked points
on a Riemann surface of arbitrary genus, we introduce the notion of a vertex
operator cluster algebra structure. Cluster elements and mutation rules are
explicitly defined, and the simplest example of a vertex operator cluster
algebra is presented.

###### Key words and phrases:

Cluster algebras; vertex operator algebras; Riemann surfaces; vertex algebra
characters; Schottky uniformization

##  1\. Introduction

The theory of cluster algebras is connected to many different areas of
mathematics, e.g., the representation theory of finite dimensional algebras,
Lie theory, Poisson geometry and Teichmüller theory[30, 31, 32]. Among these
topics are dilogarithm identities for conformal field theories[50, 51],
quantum algebras [33, 34], quivers [36, 37, 52]. Cluster algebras have
numerous applications [22, 23, 24, 25, 30, 31, 18, 38, 50, 51, 11]. Cluster
algebras appear in several applications in Conformal Field Theory [11, 50,
51]. It is natural to find an analogue of a cluster algebra structure in the
language of vertex operator algebras. In particular, a structure that
incorporates non-commutative nature of vertex operator algebra relations.

In this paper we make use of the higher genus reduction formulas for vertex
algebra characters. The recursion properties of vertex operator algebra
characters allow us to introduce an algebraic structure which we call a vertex
operator cluster algebra, simultaneously incorporating properties of cluster
algebras, non-commutative nature, and analytical and geometrical features of
character function theory of vertex operator algebras on Riemann surfaces.
Vertex operator cluster algebra seeds are defined over non-commutative
variables (elements of vertex algebra), coordinates around marked points on a
Riemann surface, and functions depending on a number of vertex operators.

In Section 5 we recall basic definitions related to cluster algebras. In
Section 2 the construction of $n$-point characters on the Schottky
reparameterization of a genus $g$ Riemann surface is reminded. A short
introduction to vertex operator algebras is given in Appendix 4. Appendix 3
contains the formulation of a vertex operator cluster algebra structure and
Proposition 2 describing the involutivity property of a vertex algebra setup.
Appendix 6 contains a description of the Schottky parameterization of a genus
$g$ Riemann surface. Appendix 7 describes auxiliary objects and matrices
needed for the Schottky genus $g$ Zhu reduction formula.

##  2\. Vertex algebra characters and Zhu reduction formula in Schottky
parameterization

In this section we recall [55] the construction and Proposition 1 for vertex
operator algebra character functions on a genus $g$ Riemann surface. In
particular, the formal partition and $n$-point correlation functions for a
vertex operator algebra associated to a genus $g$ Riemann surface
$\mathcal{S}_{g}$ are introduced in the Schottky scheme with sewing relation
(6.5). All expressions here are functions of formal variables $w_{\pm a}$,
$\rho_{a}$ and vertex operator parameters. Then we recall the genus $g$ Zhu
recursion formula with universal coefficients that have a geometrical meaning
and are meromorphic on a Riemann surface $\mathcal{S}_{g}$ for all $(w_{\pm
a},\rho_{a})\in\mathfrak{C}_{g}$ (for notations see Appendix 6). These
coefficients are generalizations of the elliptic Weierstrass functions [57].

For a $2g$ vertex algebra $V$ states

$\bm{b}=(b_{-1},b_{1};\ldots;b_{-g};b_{g}),$

and corresponding local coordinates

$\bm{w}=(w_{-1},w_{1};\ldots;w_{-g},w_{g}),$

of points $2g$ $(p_{-1},p_{1};\ldots;p_{-g},p_{g})$ on the sphere
$\mathcal{S}_{0}$, consider the genus zero $2g$-point correlation function

$\displaystyle Z^{(0)}(\bm{b,w})=$ $\displaystyle
Z^{(0)}(b_{-1},w_{-1};b_{1},w_{1};\ldots;b_{-g},w_{-g};b_{g},w_{g})$
$\displaystyle=$
$\displaystyle\prod_{a\in\mathcal{I}_{+}}\rho_{a}^{\textup{wt}(b_{a})}Z^{(0)}(\overline{b}_{1},w_{-1};b_{1},w_{1};\ldots;\overline{b}_{g},w_{-g};b_{g},w_{g}).$

where $\mathcal{I}_{+}=\\{1,2,\ldots,g\\}$. Let

$\bm{b}_{+}=(b_{1},\ldots,b_{g}),$

denote an element of a $V$-tensor product $V^{\otimes g}$-basis with dual
basis

$\bm{b}_{-}=(b_{-1},\ldots,b_{-g}),$

with respect to the bilinear form $\langle\cdot,\cdot\rangle_{\rho_{a}}$ (cf.
Appendix 4).

Let $w_{a}$ for $a\in\mathcal{I}$ be $2g$ formal variables. One identify them
with the canonical Schottky parameters (see Appendix 6). We can define the
genus $g$ partition function as

$\displaystyle
Z_{V}^{(g)}=Z_{V}^{(g)}(\bm{w,\rho})=\sum_{\bm{b}_{+}}Z^{(0)}(\bm{b,w}),$
(2.1)

for

$(\bm{w,\rho})=(w_{\pm 1},\rho_{1};\ldots;w_{\pm g},\rho_{g}).$

This definition is motivated by the sewing relation (6.5).

######  Remark 1.

Note that $Z_{V}^{(g)}$ depends on $\rho_{a}$ via the dual vectors
$\bm{b}_{-}$ as in (4.17). The genus $g$ partition function for the tensor
product $V_{1}\otimes V_{2}$ of two vertex operator algebras $V_{1}$ and
$V_{2}$ is

$Z^{(g)}_{V_{1}\otimes V_{2}}=Z_{V_{1}}^{(g)}\;Z_{V_{2}}^{(g)}.$

Now we recall a formal Zhu reduction expression for all genus $g$ Schottky
$n$-point character functions. One defines the genus $g$ formal $n$-point
function for $n$ vectors $(v_{1},\ldots,v_{n})\in V$ inserted
$(y_{1},\ldots,y_{n})$ by

$\displaystyle
Z_{V}^{(g)}(\bm{v,y})=Z_{V}^{(g)}(\bm{v,y};\bm{w,\rho})=\sum_{\bm{b}_{+}}Z^{(0)}(\bm{v,y};\bm{b,w}),$
(2.2)

where

$Z^{(0)}(\bm{v,y};\bm{b,w})=Z^{(0)}(v_{1},y_{1};\ldots;v_{n},y_{n};b_{-1},w_{-1};\ldots;b_{g},w_{g}).$

Let $U$ be a vertex operator subalgebra of $V$ where $V$ has a $U$-module
decomposition

$V=\bigoplus_{\alpha\in A}{W}_{\alpha},$

for $U$-modules ${W}_{\alpha}$ and some indexing set $A$. Let

${W}_{\bm{\alpha}}=\bigotimes_{a=1}^{g}{W}_{\alpha_{a}},$

denote a tensor product of $g$ modules

$\displaystyle
Z_{{W}_{\bm{\alpha}}}^{(g)}(\bm{v,y})=\sum_{\bm{b_{+}}\in{W}_{\bm{\alpha}}}Z^{(0)}(\bm{v,y};\bm{b,w}),$
(2.3)

where here the sum is over a basis $\\{\bm{b}_{+}\\}$ for ${W}_{\bm{\alpha}}$.
It follows that

$\displaystyle
Z_{V}^{(g)}(\bm{v,y})=\sum_{\bm{\alpha}\in\bm{A}}Z_{{W}_{\bm{\alpha}}}^{(g)}(\bm{v,y}),$
(2.4)

where the sum ranges over
$\bm{\alpha}=(\alpha_{1},\ldots,\alpha_{g})\in\bm{A}$, for
$\bm{A}=A^{\otimes{g}}$. Finally, it is useful to define corresponding formal
$n$-point correlation differential forms

$\displaystyle\mathcal{F}_{V}^{(g)}(\bm{v,y})$
$\displaystyle=Z^{(g)}(\bm{v,y})\;\bm{dy^{\textup{wt}(v)}},$
$\displaystyle\mathcal{F}_{{W}_{\bm{\alpha}}}^{(g)}(\bm{v,y})$
$\displaystyle=Z_{{W}_{\bm{\alpha}}}^{(g)}(\bm{v,y})\;\bm{dy^{\textup{wt}(v)}},$
(2.5)

where

$\bm{dy^{\textup{wt}(v)}}=\prod_{k=1}^{n}dy_{k}^{\textup{wt}(v_{k})}.$

Recall notations and identifications given in Appendix 7. Then one has:

######  Proposition 1.

The genus $g$ $(n+1)$-point formal differential
$\mathcal{F}_{{W}_{\bm{\alpha}}}^{(g)}(u,x;\bm{v,y})$ for a quasiprimary
vector $u\in U$ of weight $\textup{wt}(u)=p$ inserted at a point $p_{0}$, with
the coordinate $x$, and general vectors $(v_{1},\ldots,v_{n})$ inserted at
points $p_{1},\ldots,p_{n}$ with coordinates $(y_{1},\ldots,y_{n})$
correspondingly, respectively, satisfies the recursive identity

$\displaystyle\mathcal{F}_{{{W}_{\bm{\alpha}}}}^{(g)}(u,x;\bm{v,y})$
$\displaystyle=\sum_{k=1}^{n}\sum_{j\geq
0}\partial^{(0,j)}\;\Psi_{p}(x,y_{k})\;\mathcal{F}_{{{W}_{\bm{\alpha}}}}^{(g)}(v_{1},y_{1};\ldots;u(j)v_{k},y_{k};\ldots;v_{n},y_{n})\;dy_{k}^{j}\;$
$\displaystyle\qquad+\sum_{a=1}^{g}\Theta_{a}(x)\;O^{{W}_{\bm{\alpha}}}_{a}(u;\bm{v,y}).$
(2.6)

Here $\partial^{(0,j)}$ is given by

$\partial^{(i,j)}f(x,y)=\partial_{x}^{(i)}\partial_{y}^{(j)}f(x,y),$

for a function $f(x,y)$, and $\partial^{(0,j)}$ denotes partial derivatives
with respect to $x$ and $y_{j}$. The forms $\Psi_{p}(x,y_{k})\;dy_{k}^{j}$
given by (7.14), $\Theta_{a}(x)$ is of (7.16), and
$O^{{W}_{\bm{\alpha}}}_{a}(u;\bm{v,y})$ of (7.17).

##  3\. Schottky vertex operator cluster algebras

In this section we formulate Proposition 2 concerning a cluster vertex algebra
associated to a vertex operator algebra in the case of Schottky
parameterization of a genus $g$ Riemann surface. That Proposition clarifies a
cluster-like algebra structure for a vertex operator algebra. Let us fix a
strong-type (cf. Appendix 4) vertex operator algebra $V$. Chose $n+1$-marked
points $p_{0}$ and $p_{i}$, $i=1,\ldots,n$ on a genus $g$ Riemann surface
formed by the Schottky parameterization (cf. Appendix 6). In the vicinity of
each marked point $p_{0}$, $p_{i}$ define local coordinates $x$, $y_{i}$, with
zero at points $p_{0}$, $p_{i}$ correspondingly.

Consider $n$-tuples of arbitrary states $v_{i}\in V$, and corresponding vertex
operators

${\bf Y}(\bm{v,y})=\left(Y(v_{1},y_{1}),\ldots,Y(v_{n},y_{n})\right),$

with coordinates $(y_{1},\ldots,y_{n})$, around points $p_{i}$,
$i=1,\ldots,n$.

######  Definition 1.

We define a vertex operator cluster algebra seed

$\left({\bm{v}},{\mathbf{Y}}(\bm{v,y}),\mathcal{F}^{(g)}_{n}(\bm{v,y})\right),$
(3.1)

where

$\mathcal{F}^{(g)}_{n}(\bm{v,y})=\mathcal{F}^{(g)}_{W_{\bm{\alpha}}}(\bm{v,y}),$

(and in particular $\mathcal{F}^{(g)}_{V}(\bm{v,y})$) is a genus $g$ $n$-point
character function $\mathcal{F}^{(g)}_{n}(\bm{v,y})$ (2).

The mutation is defined as follows:

######  Definition 2.

For ${\bm{v}}$, we define the mutation ${\bm{v}}^{\prime}$ of ${\bm{v}}$ in
the direction $k\in 1,\ldots,n$ as

${\bm{v}}^{\prime}=\mu_{k}(u,m){\bm{v}}=\left(v_{1},\ldots,F_{k}(u(m)).{v_{k}},\ldots,v_{n}\right),$
(3.2)

for some $m\geq 0$, and $V$-valued functions $F_{k}(v(m))$. Note that due to
the property (4.3) we get a finite number of terms as a result of the action
of $v(m)$ on $v_{k}$, $1\leq k\leq n$. For the $n$-tuple of vertex operators
we define

$\displaystyle{\mathbf{Y}}\left({\bm{v}}^{\prime},{\bm{y}}\right)$
$\displaystyle=$
$\displaystyle\mu_{k}(u,m)\;{\mathbf{Y}}\left(\bm{v,y}\right)$ (3.3)
$\displaystyle=$
$\displaystyle\left(Y(v_{1},y_{1}),\ldots,Y(G_{k}(u(m)).v_{k},y_{k}),\ldots,Y(v_{n},y_{n})\right),$

where $G_{k}(u(m))$ are other $V$-valued functions. For $u\in V$,
$w\in\mathbb{C}$, the mutation $\mu(u,x,\bm{y})$ of
$\mathcal{F}^{(g)}_{n}(\bm{v,y})$,

$\displaystyle{\mathcal{F}^{\prime}}_{n}^{(g)}(\bm{v,y})=\mu(u,x,\bm{y})\;\mathcal{F}^{(g)}_{n}(\bm{v,y}),$
(3.4)

is defined by summation over mutations in all possible directions $k$, $1\leq
k\leq n$, with auxiliary functions $f(m,k,x,\bm{y})$, $k\in 1,\ldots,n$:

$\displaystyle{\mathcal{F}^{\prime}}^{(g)}_{n}(\bm{v,y})$ (3.5)
$\displaystyle\qquad=\sum\limits_{k=1}^{n}\sum\limits_{m\geq
0}f(m,k,x,\bm{y})\;\mathcal{F}^{(g)}_{n}(v_{1},y_{1};\ldots;H_{k}(u(m)).v_{k},y_{k};\ldots;v_{n},y_{n}),$
$\displaystyle\qquad\qquad\qquad\qquad+{\widetilde{\mathcal{F}}}^{(g)}_{n}(u,x;\bm{v,y}).$

where ${\widetilde{\mathcal{F}}}^{(g)}_{n}(u,x;\bm{v,y})$, denotes the higher
terms in the genus $g$ Zhu reduction formulas (1), and $H_{k}(u(m))$ are
$V$-valued functions. Then (3.2), (3.3), (3.5) define the mutation of the seed
(3.1).

######  Definition 3.

Definitions 1–2, the genus $g$ Zhu reduction procedure, and involutivity
condition for mutation determine the structure of a genus $g$ vertex operator
cluster algebra ${\mathcal{C}}{\mathcal{G}}^{(g)}_{n}$ of dimension $n$. We
call the full vertex operator cluster algebra the union $\bigcup_{n\geq
0}\;{\mathcal{C}}{\mathcal{G}}^{(g)}_{n}$.

######  Remark 2.

Exchange matrix of ordinary cluster algebras is replaced in this construction
with genus $g$ Schottky characters for a vertex operator algebra. These are
higher genus generalizations of matrix elements at genus zero [26], and traces
of vertex operator algebra modules at genus one [57].

Using (1), we obtain in (3.2), (3.3), and (3.5):

$\displaystyle f(m,k,x,\bm{y})$ $\displaystyle=$
$\displaystyle\partial^{(0,j)}\Psi_{p}(x,y_{k})\;dy_{k}^{j},$
$\displaystyle{\widetilde{\mathcal{F}}}^{(g)}_{n}(u,x;\bm{v,y})$
$\displaystyle=$
$\displaystyle\sum_{a=1}^{g}\Theta_{a}(x)\;O^{{W}_{\bm{\alpha}}}_{a}(u;\bm{v,y}).$
(3.6)

Next we provide an example of the Schottky vertex operator cluster algebra. We
formulate

######  Proposition 2.

For a vertex operator algebra $V$ such that $\dim V_{k}=1$, $k\in\mathbb{Z}$,
with $u=\mathbf{1}_{V}$, $w\in\mathbb{C}$, and

$F_{k}(u(m)).v=G_{k}(u(m)).v=\xi_{u,v}u(-1).v,$ $H_{k}(u(m))=u(m),$

for $m\geq 0$, and $\xi_{u,v}\in\mathbb{C}$, $\xi_{u,v}^{2}=1$, depending on
$u$ and $v$, in (3.2), (3.3), and (3.5), the mutation

$\mu=\left(\mu_{k}(\mathbf{1}_{V},-1),\mu_{k}(\mathbf{1}_{V},-1),\mu(\mathbf{1}_{V},x,\bm{y})\right),$
$\left({\bm{v}}^{\prime},{\mathbf{Y}}({\bm{v}}^{\prime},{\bm{y}}),{\mathcal{F}^{\prime}}^{(g)}_{n}({\bm{v}}^{\prime},{\bm{y}})\right)=\mu\;\left({\bm{v}},{\mathbf{Y}}(\bm{v,y}),\mathcal{F}^{(g)}_{n}(\bm{v,y})\right),$
(3.7)

defined by (3.2), (3.3), (3.5) is an involution, i.e.,

$\mu\;\mu={\rm Id}.\;$

###### Proof.

According to (4.11) for $u=\mathbf{1}_{V}\in V_{0}$, and $v_{k}\in V_{l}$,
$1\leq k\leq n$, $l\in\mathbb{Z}$,

$u(-1)u(-1).v_{k}:V_{l}\rightarrow V_{l}.$

Due to the genus $g$ Zhu reduction formula and (1), we have

$\displaystyle{\mathcal{F}^{\prime}}_{n}^{(g)}(\bm{v,y})=\mu({\mathbf{1}}_{V},x,\bm{y})\;\mathcal{F}^{(g)}_{n}(\bm{v,y})$
$\displaystyle\;=\sum\limits_{k=1}^{n}\sum\limits_{m\geq
0}f(m,k,x,\bm{y})\;\mathcal{F}^{(g)}_{n}(v_{1},z_{1};\ldots;{\mathbf{1}}_{V}[m].v_{k},z_{k};\ldots;v_{n},z_{n};\tau_{1},\tau_{2},\epsilon)$
$\displaystyle\qquad\qquad\qquad+{\widetilde{\mathcal{F}}}^{(g)}_{n}({\mathbf{1}}_{V},x;\bm{v,y})$
$\displaystyle\;=\mathcal{F}^{(g)}_{n+1}({\bf
1}_{V},x;\bm{v,y})=\mathcal{F}^{(g)}_{n}(\bm{v,y}).$ (3.8)

Thus, in this case, the mutation $\mu$ is an involution. ∎

## Acknowledgement

The author would like to thank H. V. Lê, and A. Lytchak for related
discussions. Research of the author was supported by the GACR project
18-00496S and RVO: 67985840.

##  4\. Appendix: Vertex Operator Algebras

A vertex operator algebra [8, 13, 26, 27, 35, 40, 45] is determined by a
quadruple $(V,Y,\mathbf{1}_{V},\omega)$, where is a linear space endowed with
a $\mathbb{Z}$-grading with

$V=\bigoplus_{r\in{\mathbb{Z}}}V_{r},$

with $\dim V_{r}<\infty$. The state ${\mathbf{1}}_{V}\in V_{0}$,
$\mathbf{1}_{V}\neq 0$, is the vacuum vector and $\omega\in V_{2}$ is the
conformal vector with properties described below. The vertex operator $Y$ is a
linear map

$Y:V\rightarrow\mathrm{End}(V)\left[\left[z,z^{-1}\right]\right],$

for formal variable $z$ so that for any vector $u\in V$ we have a vertex
operator

$Y(u,z)=\sum_{n\in{\mathbb{Z}}}u(n)z^{-n-1}.$ (4.1)

The linear operators (modes) $u(n):V\rightarrow V$ satisfy creativity

$Y(u,z){\mathbf{1}}_{V}=u+O(z),$ (4.2)

and lower truncation

$u(n)v=0,$ (4.3)

conditions for each $u$, $v\in V$ and $n\gg 0$. For the conformal vector
$\omega$ one has

$Y(\omega,z)=\sum_{n\in{\mathbb{Z}}}L(n)z^{-n-2},$ (4.4)

where $L(n)$ satisfies the Virasoro algebra for some central charge $C$

$[\,L(m),L(n)\,]=(m-n)L(m+n)+\frac{C}{12}(m^{3}-m)\delta_{m,-n}{\rm Id}_{V},$
(4.5)

where ${\rm Id}_{V}$ is identity operator on $V$. Each vertex operator
satisfies the translation property

$\partial_{z}Y(u,z)=Y\left(L(-1)u,z\right).$ (4.6)

The Virasoro operator $L(0)$ provides the ${\mathbb{Z}}$-grading with

$L(0)u=ru,$

for $u\in V_{r}$, $r\in{\mathbb{Z}}$. Finally, the vertex operators satisfy
the Jacobi identity

$\displaystyle
z_{0}^{-1}\delta\left(\frac{z_{1}-z_{2}}{z_{0}}\right)Y(u,z_{1})Y(v,z_{2})-z_{0}^{-1}\delta\left(\frac{z_{2}-z_{1}}{-z_{0}}\right)Y(v,z_{2})Y(u,z_{1})$
$\displaystyle\qquad\qquad\qquad=z_{2}^{-1}\delta\left(\frac{z_{1}-z_{0}}{z_{2}}\right)Y\left(Y(u,z_{0})v,z_{2}\right).$
(4.7)

These axioms imply locality, skew-symmetry, associativity and commutativity
conditions:

$\displaystyle(z_{1}-z_{2})^{N}Y(u,z_{1})Y(v,z_{2})=(z_{1}-z_{2})^{N}Y(v,z_{2})Y(u,z_{1}),$
(4.8) $\displaystyle Y(u,z)v=e^{zL(-1)}Y(v,-z)u,$
$\displaystyle(z_{0}+z_{2})^{N}Y(u,z_{0}+z_{2})Y(v,z_{2})w=(z_{0}+z_{2})^{N}Y(Y(u,z_{0})v,z_{2})w,$
$\displaystyle u(k)Y(v,z)-Y(v,z)u(k)=\sum\limits_{j\geq 0}\left(k\atop
j\right)Y(u(j)v,z)z^{k-j},$ (4.9)

for $u$, $v$, $w\in V$ and integers $N\gg 0$. For $v={\mathbf{1}}_{V}$ one has

$Y({\mathbf{1}}_{V},z)={\rm Id}_{V}.$ (4.10)

Note also that modes of homogeneous states are graded operators on $V$, i.e.,
for $v\in V_{k}$,

$\displaystyle v(n):V_{m}\rightarrow V_{m+k-n-1}.$ (4.11)

In particular, let us define the zero mode $o(v)$ of a state of weight
$wt(v)=k$, i.e., $v\in V_{k}$, as

$o(v)=v(wt(v)-1),$ (4.12)

extending to $V$ additively.

######  Definition 4.

Given a vertex operator algebra $V$, one defines the adjoint vertex operator
with respect to $\alpha\in\mathbb{C}$, by

$\displaystyle Y^{\dagger}(u,z)$ $\displaystyle=$
$\displaystyle\sum_{n\in\mathbb{Z}}u^{\dagger}(n)z^{-n-1}$ (4.13)
$\displaystyle=$ $\displaystyle
Y\left(\exp\left(\frac{z}{\alpha}L(1)\right)\left(-\frac{\alpha}{z^{2}}\right)^{L(0)}u,\frac{\alpha}{z}\right),$

associated with the formal Möbius map [26]

$z\mapsto\frac{\alpha}{z}.$

######  Definition 5.

An element $u\in V$ is called quasiprimary if

$L(1)u=0.$

For quasiprimary $u$ of weight $\textup{wt}(u)$ one has

$\displaystyle
u^{\dagger}(n)=(-1)^{\textup{wt}(u)}\alpha^{n+1-\textup{wt}(u)}u(2\textup{wt}(u)-n-2).$

######  Definition 6.

A bilinear form

$\langle.,.\rangle:V\times V\rightarrow\mathbb{C},$

is called invariant if [26, 43]

$\displaystyle\langle Y(u,z)a,b\rangle=\langle a,Y^{\dagger}(u,z)b\rangle,$
(4.14)

for all $a$, $b$, $u\in V$.

Notice that the adjoint vertex operator $Y^{\dagger}(.,.)$ as well as the
bilinear form $\langle.,.\rangle$, depend on $\alpha$. In terms of modes, we
have

$\displaystyle\langle u(n)a,b\rangle=\langle a,u^{\dagger}(n)b\rangle.$ (4.15)

Choosing $u=\omega$, and for $n=1$ implies

$\langle L(0)a,b\rangle=\langle a,L(0)b\rangle.$

Thus,

$\langle a,b\rangle=0,$

when $\textup{wt}(a)\neq\textup{wt}(b)$.

######  Definition 7.

A vertex operator algebra is called of strong-type if

$V_{0}=\mathbb{C}\mathbf{1}_{V},$

and $V$ is simple and self-dual, i.e., $V$ is isomorphic to the dual module
$V^{\prime}$ as a $V$-module.

It is proven in [43] that a strong-type vertex operator algebra $V$ has a
unique invariant non-degenerate bilinear form up to normalization. This
motivates

######  Definition 8.

The form $\langle.,.\rangle$ on a strong-type vertex operator algebra $V$ is
the unique invariant bilinear form $\langle.,.\rangle$ normalized by

$\langle\mathbf{1}_{V},\mathbf{1}_{V}\rangle=1.$

Given a vertex operator algebra $(V,Y(.,.),\mathbf{1},\omega)$, one can find
an isomorphic vertex operator algebra
$(V,Y[.,.],\mathbf{1},\widetilde{\omega})$ called [57] the square-bracket
vertex operator algebra. Both algebras have the same underlying vector space
$V$, vacuum vector $\mathbf{1}_{V}$, and central charge. The vertex operator
$Y[.,.]$ is determined by

$\displaystyle
Y[v,z]=\sum_{n\in\mathbb{Z}}v[n]z^{-n-1}=Y\left(q_{z}^{L(0)}v,q_{z}-1\right).$

The new square-bracket conformal vector is

$\widetilde{\omega}=\omega-\frac{c}{24}\mathbf{1},$

with the vertex operator

$Y[\widetilde{\omega},z]=\sum_{n\in\mathbb{Z}}L[n]z^{-n-2}.$

The square-bracket Virasoro operator mode $L[0]$ provides an alternative
$\mathbb{Z}$–grading on $V$, i.e., $\textup{wt}[v]=k$ if

$L[0]v=kv,$

where $\textup{wt}[v]=\textup{wt}(v)$ for primary $v$, and $L(n)v=0$ for all
$n>0$. We can similarly define a square-bracket bilinear form
$\langle.,.\rangle_{\mathrm{sq}}$.

Next we recall a lemma from [55]. The bilinear form
$\langle\cdot,\cdot\rangle$ of is invertible and that

$\langle u,v\rangle=0,$

Let $\\{b\\}$ be a homogeneous basis for $V$ with the dual basis
$\\{\overline{b}\\}$.

######  Lemma 1.

For $u$ quasiprimary of weight $p$ we have

$\displaystyle\sum_{b\in
V_{n}}\left(u(m)b\right)\otimes\overline{b}=\sum_{b\in
V_{n+p-m-1}}b\otimes\left(u^{\dagger}(m)\overline{b}\right).$ (4.16)

######  Remark 3.

Suppose that $U$ is a vertex operator subalgebra of $V$ and ${W}\subset V$ is
a $U$-module. For $u\in U$ and homogeneous ${W}$-basis $\\{w\\}$ we may then
extend (4.16) to obtain

$\displaystyle\sum_{w\in{W}_{n}}\left(u(m)w\right)\otimes\overline{w}=\sum_{w\in{W}_{n+p-m-1}}w\otimes\left(u^{\dagger}(m)\overline{w}\right).$

For the Schottky setup we have the following properties associated to the
$\rho$-sewing. For each $a\in\mathcal{I}_{+}$, let $\\{b_{a}\\}$ denote a
homogeneous $V$-basis and let $\\{\overline{b}_{a}\\}$ be the dual basis with
$\langle\cdot,\cdot\rangle_{1}$, i.e., with $\rho=1$. Define

$\displaystyle b_{-a}=\rho_{a}^{\textup{wt}(b_{a})}\overline{b}_{a},\quad
a\in\mathcal{I}_{+},$ (4.17)

for a formal $\rho_{a}$. We then identify $\rho_{a}$ with a Schottky sewing
parameter. Then $\\{b_{-a}\\}$ is a dual basis for the bilinear form
$\langle\cdot,\cdot\rangle_{\rho_{a}}$ with adjoint modes

$\displaystyle u^{\dagger}_{\rho_{a}}(m)=(-1)^{p}\rho_{a}^{m-p+1}u(2p-2-m),$
(4.18)

for $u$ quasiprimary of weight $p$.

##  5\. Appendix: Definition of a cluster algebra

Let us first recall the notion of a cluster algebra [19, 20, 21] following
of[54]. We consider commutative cluster algebras of rank $n$. The set of all
cluster variables is constructed recursively from an initial set of $n$
cluster variables using mutations. Every mutation defines a new cluster
variable as a rational function of the cluster variables constructed
previously. Thus, recursively, every cluster variable is a certain rational
function in the initial $n$ cluster variables. These rational functions are
Laurent polynomials [19].

A cluster algebra is determined by its initial seed which consists of a
cluster

${\bf x}=(x_{1},\ldots,x_{n}),$

of algebraically independent set of generators, a coefficient tuple

${\bf y}=(y_{1},\ldots,y_{n}),$

and a skew-symmetrizable $n\times n$ integer exchange matrix

$B=\left(b_{ij}\right),$

i.e., $b_{i,j}=-b_{j,i}$. The coefficients
$\left\\{y_{1},\ldots,y_{n}\right\\}$ are taken in a torsion free abelian
group ${\mathbb{P}}$. The mutation in direction $k$ defines a new cluster

$x^{\prime}_{k}x_{k}=y^{+}\prod\limits_{b_{k,i}>0}x_{i}^{b_{k,i}}+y^{-}\prod\limits_{b_{k,i}<0}x_{i}^{-b_{k,i}},$
(5.1)

where $y^{\pm}$ are certain monomials in $(y_{1},\ldots,y_{n})$. Mutations
also transform the coefficient tuple $y$ and the matrix $B$.

If $\zeta$ is any cluster variable, then $u$ is obtained from the initial
cluster $\bm{x}$ by a sequence of mutations, then [19] $\zeta$ can be written
as a Laurent polynomial in variables $(x_{1},\ldots,x_{n})$, that is,

$f(\bm{x})=\zeta\;{\prod\limits_{i=1}^{n}x_{i}^{d_{i}}},$ (5.2)

for some $d_{i}$, where $f(\bm{x})$ is a polynomial with coefficients in the
group ring ${{\mathbb{Z}}{\mathbb{P}}}$ of the coefficient group
${\mathbb{P}}$. A cluster algebra is of finite type if it has only a finite
number of seeds. In[20] it was shown that cluster algebras of finite type can
be classified in terms of the Dynkin diagrams of finite-dimensional simple Lie
algebras.

###  5.1. Formal definition

Let ${\mathbb{P}}$ be an abelian group with binary operation $\oplus$,
${\mathbb{Z}}{\mathbb{P}}$ be the group ring of ${\mathbb{P}}$, and let
${\mathbb{Q}}{\mathbb{P}}(\bm{x})$ be the field of rational functions in $n$
variables with coefficients in ${\mathbb{Q}}{\mathbb{P}}$.

######  Definition 9.

A seed is a triple $({\mathbf{x}},{\mathbf{y}},B)$, where
${\mathbf{x}}=\left\\{x_{1},\ldots,x_{n}\right\\}$ is a basis of
${\mathbb{Q}}{\mathbb{P}}\left(x_{1},\ldots,x_{n}\right)$,
${\mathbf{y}}=\left\\{y_{1},\ldots,y_{n}\right\\}$, is an $n$-tuple of
elements $y_{i}\in{\mathbb{P}}$, and $B$ is a skew-symmetrizable matrix.

######  Definition 10.

Given a seed

$\left({\mathbf{x}},{\mathbf{y}},B\right),$

its mutation $\mu_{k}({\mathbf{x}},{\mathbf{y}},B)$ in direction $k$ is a new
seed $({\mathbf{x}}^{\prime},{\mathbf{y}}^{\prime},B^{\prime})$ defined as
follows. Let $[x]_{+}=max(x,0)$. Then we have $B^{\prime}=(b^{\prime}_{ij})$
with

$\displaystyle b^{\prime}_{ij}=\left[\begin{array}[]{l}b_{ij}\;{\rm
for}\;i=k\;{\rm or}\;j=k,\\\
b_{ij}+[-b_{ik}]_{+}b_{kj}+b_{ik}[b_{kj}]_{+},\;\;{\rm
otherwise}.\end{array}\right.$ (5.5)

For new coefficients
${\mathbf{y}}^{\prime}=\left(y^{\prime}_{1},\ldots,y^{\prime}_{n}\right)$,
with

$\displaystyle y^{\prime}_{j}=\left[\begin{array}[]{l}y^{-1}_{k}\;{\rm
if}\;j=k,\\\ y_{j}y_{k}^{[b_{kj}]_{+}}(y_{k}\oplus 1)^{-b_{kj}}\;{\rm
if}\;j\neq k,\end{array}\right.$ (5.8)

and ${\bf x}=(x_{1},\ldots,x_{n})$, where

$(y_{k}\oplus
1)\;x_{k}\;x^{\prime}_{k}=y_{k}\prod\limits_{i=1}^{n}x_{i}^{[b_{ik}]_{+}}+\prod\limits_{i=1}^{n}x_{i}^{[-b_{ik}]_{+}}.$
(5.9)

Mutations are involutions, i.e.,

$\mu_{k}\;\mu_{k}\;({\bf x},{\bf y},B)=({\bf x},{\bf y},B).$

##  6\. Appendix: The Schottky uniformization of Riemann surfaces

In this appendix we recall the Schottky uniformization of Riemann surfaces
[55]. Consider a compact marked Riemann surface $\mathcal{S}_{g}$ of genus
$g$, e.g., [17, 47, 15, 7], with canonical homology basis $\alpha_{a}$,
$\beta_{a}$ for $a\in\mathcal{I}_{+}=\\{1,2,\ldots,g\\}$. We recall the
construction of a genus $g$ Riemann surface $\mathcal{S}_{g}$ using the
Schottky uniformization where we sew $g$ handles to the Riemann sphere

$\mathcal{S}_{0}\cong\widehat{\mathbb{C}}=\mathbb{C}\bigcup\\{\infty\\},$

e.g., [29, 7]. Every Riemann surface can be non-uniquely Schottky uniformized
[6]. For $a\in\mathcal{I}=\\{\pm 1,\pm 2,\ldots,\pm g\\}$, let
$\mathcal{C}_{a}\subset\mathcal{S}_{0}$ be $2g$ non-intersecting Jordan
curves. For $z\in\mathcal{C}_{a}$, $z^{\prime}\in\mathcal{C}_{-a}$, $W_{\pm
a}\in\widehat{\mathbb{C}}$, $a\in\mathcal{I}_{+}$, and $q_{a}$ with

$0<|q_{a}|<1,$

let curves be identified by the sewing relation

$\displaystyle\frac{z^{\prime}-W_{-a}}{z^{\prime}-W_{a}}\cdot\frac{z-W_{a}}{z-W_{-a}}=q_{a}.$
(6.1)

For $a\in\mathcal{I}_{+}$, Introduce

$\displaystyle\sigma_{a}=(W_{-a}-W_{a})^{-1/2}\begin{pmatrix}1&-W_{-a}\\\
1&-W_{a}\end{pmatrix},$ (6.2)

and

$\displaystyle\gamma_{a}=\sigma_{a}^{-1}\begin{pmatrix}q_{a}^{1/2}&0\\\
0&q_{a}^{-1/2}\end{pmatrix}\sigma_{a},$ (6.3)

Thus

$z^{\prime}=\gamma_{a}z.$

Note that

$\sigma_{a}(W_{-a})=0,$

and

$\sigma_{a}(W_{a})=\infty,$

are, respectively, attractive and repelling fixed points of the map

$Z\rightarrow Z^{\prime}=q_{a}Z,$

for

$Z=\sigma_{a}z,$

and

$Z^{\prime}=\sigma_{a}z^{\prime}.$

Here $W_{-a}$ and $W_{a}$ are the corresponding fixed points for $\gamma_{a}$.
One identifies the standard homology cycles $\alpha_{a}$ with
$\mathcal{C}_{-a}$ and $\beta_{a}$ with a path connecting
$z\in\mathcal{C}_{a}$ to

$z^{\prime}=\gamma_{a}z,\in\mathcal{C}_{-a}.$

and $z^{\prime}\in\mathcal{C}_{-a}$.

######  Definition 11.

The genus $g$ Schottky group $\Gamma$ is the free group with generators
$\gamma_{a}$. Define

$\gamma_{-a}=\gamma_{a}^{-1}.$

The independent elements of $\Gamma$ are reduced words of length $k$ of the
form

$\gamma=\gamma_{a_{1}}\ldots\gamma_{a_{k}},$

where $a_{i}\neq-a_{i+1}$ for each $i=1,\ldots,k-1$.

Let $\Lambda(\Gamma)$ denote the limit set of $\Gamma$, i.e., the set of limit
points of the action of $\Gamma$ on $\widehat{\mathbb{C}}$. Then

$\mathcal{S}_{g}\simeq\Omega_{0}/\Gamma$

where

$\Omega_{0}=\widehat{\mathbb{C}}-\Lambda(\Gamma).$

We let $\mathcal{D}\subset\widehat{\mathbb{C}}$ denote the standard connected
fundamental region with oriented boundary curves $\mathcal{C}_{a}$. Define

$w_{a}=\gamma_{-a}.\infty.$

Using (6.1) we find

$\displaystyle w_{a}=\frac{W_{a}-q_{a}W_{-a}}{1-q_{a}},$ (6.4)

for $a\in\mathcal{I}$. where we define $q_{-a}=q_{a}$. Then (6.1) is
equivalent to

$\displaystyle(z^{\prime}-w_{-a})(z-w_{a})=\rho_{a},$ (6.5)

with

$\displaystyle\rho_{\pm a}=-\frac{q_{a}(W_{a}-W_{-a})^{2}}{(1-q_{a})^{2}}.$
(6.6)

(6.5) implies

$\displaystyle\gamma_{a}z=w_{-a}+\frac{\rho_{a}}{z-w_{a}}.$ (6.7)

Let $\Delta_{a}$ be the disc with centre $w_{a}$ and radius
$|\rho_{a}|^{\frac{1}{2}}$. One chooses the Jordan curve $\mathcal{C}_{a}$ to
be the boundary of $\Delta_{a}$. Then $\gamma_{a}$ maps the exterior
(interior) of $\Delta_{a}$ to the interior (exterior) of $\Delta_{-a}$ since

$\displaystyle|\gamma_{a}z-w_{-a}||z-w_{a}|=|\rho_{a}|.$

The discs $\Delta_{a}$, $\Delta_{b}$ are non-intersecting if and only if

$\displaystyle|w_{a}-w_{b}|>|\rho_{a}|^{\frac{1}{2}}+|\rho_{b}|^{\frac{1}{2}},$
(6.8)

for all $a\neq b$. One defines $\mathfrak{C}_{g}$ to be the set

$\\{(w_{a},w_{-a},\rho_{a})|a\in\mathcal{I}_{+}\\}\subset\mathbb{C}^{3g},$

satisfying (6.8). We refer to $\mathfrak{C}_{g}$ as the Schottky parameter
space.

The relation (6.1) is Möbius invariant for

$\gamma=\left(\begin{array}[]{cc}A&B\\\
C&D\end{array}\right)\in\textup{SL}_{2}(\mathbb{C}),$

with

$(z,z^{\prime},W_{a},q_{a})\rightarrow(\gamma z,\gamma z^{\prime},\gamma
W_{a},q_{a}),$

giving an $\textup{SL}_{2}(\mathbb{C})$ action on $\mathfrak{C}_{g}$ as
follows

$\displaystyle\gamma:(w_{a},\rho_{a})\mapsto$
$\displaystyle\qquad\left(\frac{\left(Aw_{a}+B\right)\left(Cw_{-a}+D\right)-\rho_{a}\,AC}{\left(Cw_{a}+D\right)\left(Cw_{-a}+D\right)-\rho_{a}\,{C}^{2}},{\frac{\rho_{a}}{\left(\left(Cw_{a}+D\right)\left(Cw_{-a}+D\right)-\rho_{a}\,{C}^{2}\right)^{2}}}\right).$
(6.9)

######  Definition 12.

One defines the Schottky space as

$\mathfrak{S}_{g}=\mathfrak{C}_{g}/\textup{SL}_{2}(\mathbb{C}),$

This provides a natural covering space for the moduli space of genus $g$
Riemann surfaces (of dimension 1 for $g=1$ and $3g-3$ for $g\geq 2$).

##  7\. Appendix: coefficient functions in the Zhu reduction formula

For purposes of the formula (1) we recall here certain definitions [55].
Define a column vector

$X=(X_{a}(m)),$

indexed by $m\geq 0$ and $a\in\mathcal{I}$ with components

$\displaystyle
X_{a}(m)=\rho_{a}^{-\frac{m}{2}}\sum_{\bm{b}_{+}}Z^{(0)}(\ldots;u(m)b_{a},w_{a};\ldots),$
(7.1)

and a row vector

$p(x)=(p_{a}(x,m)),$

for $m\geq 0,a\in\mathcal{I}$ with components

$\displaystyle
p_{a}(x,m)=\rho_{a}^{\frac{m}{2}}\partial^{(0,m)}\psi_{p}^{(0)}(x,w_{a}).$
(7.2)

Introduce the column vector

$G=(G_{a}(m)),$

for $m\geq 0,a\in\mathcal{I}$, given by

$\displaystyle G=\sum_{k=1}^{n}\sum_{j\geq
0}\partial_{k}^{(j)}\;q(y_{k})\;Z_{V}^{(g)}(v_{1},y_{1};\ldots;u(j)v_{k},y_{k};\ldots;v_{n},y_{n}),$

where $q(y)=(q_{a}(y;m))$, for $m\geq 0$, $a\in\mathcal{I}$, is a column
vector with components

$\displaystyle
q_{a}(y;m)=(-1)^{p}\rho_{a}^{\frac{m+1}{2}}\partial^{(m,0)}\psi_{p}^{(0)}(w_{-a},y),$
(7.3)

and

$R=(R_{ab}(m,n)),$

for $m$, $n\geq 0$ and $a$, $b\in\mathcal{I}$ is a doubly indexed matrix with
components

$\displaystyle
R_{ab}(m,n)=\begin{cases}(-1)^{p}\rho_{a}^{\frac{m+1}{2}}\rho_{b}^{\frac{n}{2}}\partial^{(m,n)}\psi_{p}^{(0)}(w_{-a},w_{b}),&a\neq-b,\\\
(-1)^{p}\rho_{a}^{\frac{m+n+1}{2}}\mathcal{E}_{m}^{n}(w_{-a}),&a=-b,\end{cases}$
(7.4)

where

$\displaystyle\mathcal{E}_{m}^{n}(y)=\sum_{\ell=0}^{2p-2}\partial^{(m)}f_{\ell}(y)\;\partial^{(n)}y^{\ell},$
(7.5)
$\displaystyle\psi_{p}^{(0)}(x,y)=\frac{1}{x-y}+\sum_{\ell=0}^{2p-2}f_{\ell}(x)y^{\ell},$
(7.6)

for any Laurent series $f_{\ell}(x)$ for $\ell=0,\ldots,2p-2$. Define the
doubly indexed matrix $\Delta=(\Delta_{ab}(m,n))$ by

$\displaystyle\Delta_{ab}(m,n)=\delta_{m,n+2p-1}\delta_{ab}.$ (7.7)

Denote by

$\widetilde{R}=R\Delta,$

and the formal inverse $(I-\widetilde{R})^{-1}$ is given by

$\displaystyle\left(I-\widetilde{R}\right)^{-1}=\sum_{k\geq
0}\widetilde{R}^{\,k}.$ (7.8)

Define $\chi(x)=(\chi_{a}(x;\ell))$ and

$o(u;\bm{v,y})=(o_{a}(u;\bm{v,y};\ell)),$

are finite row and column vectors indexed by $a\in\mathcal{I}$, $0\leq\ell\leq
2p-2$ with

$\displaystyle\chi_{a}(x;\ell)$
$\displaystyle=\rho_{a}^{-\frac{\ell}{2}}(p(x)+\widetilde{p}(x)(I-\widetilde{R})^{-1}R)_{a}(\ell),$
(7.9) $\displaystyle o_{a}(\ell)$
$\displaystyle=o_{a}(u;\bm{v,y};\ell)=\rho_{a}^{\frac{\ell}{2}}X_{a}(\ell),$
(7.10)

and where

$\widetilde{p}(x)=p(x)\Delta.$

$\psi_{p}(x,y)$ is defined by

$\displaystyle\psi_{p}(x,y)=\psi_{p}^{(0)}(x,y)+\widetilde{p}(x)(I-\widetilde{R})^{-1}q(y).$
(7.11)

For each $a\in\mathcal{I}_{+}$ we define a vector

$\theta_{a}(x)=(\theta_{a}(x;\ell)),$

indexed by $0\leq\ell\leq 2p-2$ with components

$\displaystyle\theta_{a}(x;\ell)=\chi_{a}(x;\ell)+(-1)^{p}\rho_{a}^{p-1-\ell}\chi_{-a}(x;2p-2-\ell).$
(7.12)

Now define the following vectors of formal differential forms

$\displaystyle P(x)=p(x)\;dx^{p},$ $\displaystyle Q(y)=q(y)\;dy^{1-p},$ (7.13)

with

$\widetilde{P}(x)=P(x)\Delta.$

Then with

$\Psi_{p}(x,y)=\psi_{p}(x,y)\;dx^{p}\;dy^{1-p},$ (7.14)

we have

$\displaystyle\Psi_{p}(x,y)=\Psi_{p}^{(0)}(x,y)+\widetilde{P}(x)(I-\widetilde{R})^{-1}Q(y).$
(7.15)

Defining

$\Theta_{a}(x;\ell)=\theta_{a}(x;\ell)\;dx^{p},$ (7.16)

and

$O_{a}(u;\bm{v,y};\ell)=o_{a}(u;\bm{v,y};\ell)\;\bm{dy^{\textup{wt}(v)}},$
(7.17)

######  Remark 4.

The $\Theta_{a}(x)$, and $\Psi_{p}(x,y)$ coefficients depend on
$p=\textup{wt}(u)$ but are otherwise independent of the vertex operator
algebra $V$. Note that for a 1-point function, (1) implies

$\displaystyle\mathcal{F}_{V}^{(g)}(u,x)$
$\displaystyle=\sum_{a=1}^{g}\Theta_{a}(x)\;O_{a}(u).$ (7.18)

## References

  * [1] Ahlfors, L. Some remarks on Teichmüller’s space of Riemann surfaces, Ann.Math. 74 (1961) 171-191.
  * [2] Alvarez-Gaume, L., Moore, G. and Vafa, C. Theta functions, modular invariance, and strings, Comm.Math.Phys. 106 1-?40 (1986).
  * [3] Baker, H.F. Abel’s Theorem and the Allied Theory Including the Theory of Theta Functions, Cambridge University Press (Cambridge, 1995).
  * [4] Belavin, A., Polyakov, A. and Zamolodchikov, A.: Infinite conformal symmetry in two-dimensional quantum field theory. Nucl. Phys. B241 333–380 (1984).
  * [5] Bers, L. Inequalities for finitely generated Kleinian groups, J.Anal.Math. 18 23–41 (1967).
  * [6] Bers, L. Automorphic forms for Schottky groups, Adv.Math. 16 332?-361 (1975).
  * [7] Bobenko, A. Introduction to compact Riemann surfaces, in Computational Approach to Riemann Surfaces, edited Bobenko, A. and Klein, C., Springer-Verlag (Berlin, Heidelberg, 2011).
  * [8] Borcherds, R.E.: Vertex algebras, Kac-Moody algebras and the monster. Proc. Nat. Acad. Sc. 83, 3068–3071 (1986).
  * [9] Burnside, W. On a class of automorphic functions, Proc.L.Math.Soc. 23 49–88 (1891).
  * [10] Codogni, G. Vertex algebras and Teichmuller modular forms, arXiv:1901.03079.
  * [11] Ph. Di Francesco and R. Kedem, Q-systems as cluster algebras. II. Lett. Math. Phys. 89, no. 3, (2009) 183.
  * [12] L. Dolan, P. Goddard and P. Montague, Conformal field theories, representations and lattice constructions, Commun. Math. Phys. 179 (1996) p. 61.
  * [13] C. Dong and J. Lepowsky, Generalized Vertex Algebras and Relative Vertex Operators (Progress in Math. Vol. 112, Birkhäuser, Boston, 1993).
  * [14] Eguchi, T. and Ooguri, H. Conformal and current algebras on a general Riemann surface, Nucl. Phys. B282 308–328 (1987).
  * [15] Fay, J.D. Theta Functions on Riemann Surfaces, Lecture Notes in Mathematics, Vol. 352. Springer-Verlag, (Berlin-New York, 1973).
  * [16] Ph. Di Francesco, P., Mathieu, P. and Senechal, D.: Conformal Field Theory. Springer Graduate Texts in Contemporary Physics, Springer-Verlag, New York (1997).
  * [17] H.M. Farkas and I. Kra, Riemann surfaces, (Springer-Verlag, New York, 1980).
  * [18] S. Fomin, M. Shapiro, D. Thurston, Cluster algebras and Acta Mathematica 201 (2008) 83.
  * [19] S. Fomin, A. Zelevinsky, Cluster algebras. I. Foundations, J. of the Amer. Math. Soc. (2002) 15 (2) 49. 7–529 .
  * [20] S. Fomin, A. Zelevinsky. Cluster algebras. II. Finite type classification, Inventiones Mathematicae 154 (1): (2003) 63–121 (2003).
  * [21] S. Fomin, A. Zelevinsky. Cluster algebras. IV. Coefficients, Compositio Mathematica 143 (1): (2007) p. 112–164 (2007).
  * [22] V. Fock and A. Goncharov, Moduli spaces of local systems and higher Teichmüller theory. Publ. Math. Inst. Hautes Études Sci. (2006) 103, 1.
  * [23] V. Fock and A. Goncharov, Cluster ensembles, quantization and the dilogarithm, Ann. Sci. Éc. Norm. Supér. (4) 42, no. 6, (2009) 865.
  * [24] V. Fock and A. Goncharov, Cluster ensembles, quantization and the dilogarithm. II, in The intertwiner. Algebra, arithmetic, and geometry: in honor of Yu. I. Manin. Vol. I, (Progr. Math., 269, Birkhäuser Boston, Inc., Boston, MA, 2009) p. 655.
  * [25] V. Fock and A. Goncharov, Dual Teichmüller and lamination spaces, in Handbook of Teichmüller theory. Vol. I, IRMA Lect. Math. Theor. Phys., 11 (Eur. Math. Soc., Zürich, 2007), p. 647.
  * [26] I. Frenkel,Y.-Z. Huang and J. Lepowsky, On Axiomatic Approaches to Vertex Operator Algebras and Modules, Mem. AMS. 104 No. 494 (1993).
  * [27] I. Frenkel, J. Lepowsky and A. Meurman, Vertex Operator Algebras and the Monster, Pure and Appl. Math. Vol. 134 (Academic Press, Boston, 1988).
  * [28] Freidan, D. and Shenker, S.: The analytic geometry of two-dimensional conformal field theory. Nucl. Phys. B281 509–545 (1987).
  * [29] Ford, L.R. Automorphic Functions, AMS-Chelsea, (Providence, 2004).
  * [30] M. Gekhtman, M. Shapiro and A. Vainshtein, Mosc. Math. J. 3, no. 3, (2003) 899.
  * [31] M. Gekhtman, M. Shapiro and A. Vainshtein, Duke Math. J. 127, no. 2, (2005) 291.,
  * [32] M. Gekhtman, M. Shapiro, and A. Vainshtein. Cluster algebras and Poisson geometry, in Mathematical Surveys and Monographs, vol. 167 (American Mathematical Society, Providence, RI, 2010).
  * [33] Ch. Geiss, B. Leclerc and J. Schröer. Cluster structures on Selecta Math. (N.S.) 19, no. 2, (2013) 337.
  * [34] D.Hernandez and B. Leclerc. Cluster algebras and quantum affine algebras, Duke Math. J. 154, no. 2, (2010) 265.
  * [35] V. Kac, Vertex Algebras for Beginners, Second Ed. (Univ. Lect. Ser. 10, AMS, 1998).
  * [36] B. Keller, Cluster algebras, quiver representations and triangulated categories, Triangulated categories”, 76–160, in London Math. Soc. Lecture Note Ser., (Cambridge Univ. Press, Cambridge, 2010) p. 375.
  * [37] B. Keller, Quantum loop algebras, quiver varieties, and cluster algebras, in Representations of algebras and related topics, (EMS Ser. Congr. Rep., Eur. Math. Soc., Zürich, 2011) p. 117.
  * [38] M. Kontsevich and Y. Soibelman, Stability structures, Donaldson- Thomas invariants and cluster transformations, arXiv:0811.2435, 2008.
  * [39] Knizhnik, V.G.: Multiloop amplitudes in the theory of quantum strings and complex geometry. Sov. Phys. Usp. 32 945–971 (1989).
  * [40] J. Lepowsky and H. Li, Introduction to Vertex Algebras, (Progress in Math. Vol. 227, Birkhäuser, Boston, 2004).
  * [41] S. Lang, Introduction to Modular Forms (Springer, Berlin, 1976).
  * [42] Lang, S.: Elliptic functions. Springer-Verlag, New York (1987).
  * [43] Li, H.: Symmetric invariant bilinear forms on vertex operator algebras. J. Pure. Appl. Alg. 96, 279–297 (1994).
  * [McI] McIntyre, A. Analytic torsion and Faddeev-Popov ghosts, SUNY PhD thesis 2002, hdl.handle.net/11209/10688.
  * [44] McIntyre, A. and Takhtajan, L.A. Holomorphic factorization of determinants of Laplacians on Riemann surfaces and a higher genus generalization of Kronecker’s first limit formula, GAFA, Geom.Funct.Anal. 16 1291–1323 (2006).
  * [45] A. Matsuo and K. Nagatomo, Axioms for a Vertex Algebra and the Locality of Quantum Fields, (Math. Soc. of Japan Memoirs Vol. 4, Tokyo, 1999).
  * [46] M. Miyamoto, Modular invariance of vertex operator algebras satisfying $C_{2}$-cofiniteness, Duke Math. J. Vol. 122 No. 1 (2004) p. 51.
  * [47] Mumford, D. Tata Lectures on Theta I and II, Birkhäuser, (Boston, 1983).
  * [48] Odesskii, A. Deformations of complex structures on Riemann surfaces and integrable structures of Whitham type hierarchies, arXiv:1505.07779.
  * [49] Rauch, H.E. On the transcendental moduli of algebraic Riemann surfaces, Proc. Nat. Acad. Sc. 11 42–48 (1955).
  * [50] T. Nakanishi, Dilogarithm identities for conformal field theories and cluster algebras: simply laced case, Nagoya Math. J. 202, (2011) 23–43 .
  * [51] T. Nakanishi, Periodicities in cluster algebras and dilogarithm identities, in Representations of algebras and related topics, (EMS Ser. Congr. Rep., Eur. Math. Soc., Zürich, 2011) p. 407.
  * [52] H. Nakajima, Quiver varieties and cluster algebras, Kyoto J. Math.51 no. 1, (2011) p. 71–126 (2011).
  * [53] J.-P. Serre, A Course in Arithmetic (Springer, New York, 1973).
  * [54] R. Schiffler, On cluster algebras arising from unpunctured surfaces. II. Adv. Math. 223 no. 6 (2010) 1885.
  * [55] M. P. Tuite, M. Welby. General Genus Zhu Recursion for Vertex Operator Algebras, arXiv:1911.06596.
  * [56] Yamada, A.: Precise variational formulas for abelian differentials. Kodai. Math. J. 3, 114–143 (1980).
  * [57] Zhu, Y. Modular-invariance of characters of vertex operator algebras, J.Amer.Math.Soc. 9 237–302 (1996).