# On the Complementarity between Pre-Training and Back-Translation
for Neural Machine Translation

Xuebo Liu1 , Longyue Wang2, Derek F. Wong1, Liang Ding3,
Lidia S. Chao1, Shuming Shi2 and Zhaopeng Tu2
1NLP2CT Lab, Department of Computer and Information Science, University of
Macau
2Tencent AI Lab 3The University of Sydney
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Work was done when Xuebo Liu and Liang Ding were
interning at Tencent AI Lab.

###### Abstract

Pre-training (PT) and back-translation (BT) are two simple and powerful
methods to utilize monolingual data for improving the model performance of
neural machine translation (NMT). This paper takes the first step to
investigate the complementarity between PT and BT. We introduce two probing
tasks for PT and BT respectively and find that PT mainly contributes to the
encoder module while BT brings more benefits to the decoder. Experimental
results show that PT and BT are nicely complementary to each other,
establishing state-of-the-art performances on the WMT16 English-Romanian and
English-Russian benchmarks. Through extensive analyses on sentence originality
and word frequency, we also demonstrate that combining Tagged BT with PT is
more helpful to their complementarity, leading to better translation quality.
Source code is freely available at https://github.com/SunbowLiu/PTvsBT.

##  1 Introduction

Neural machine translation (Bahdanau et al., 2015; Gehring et al., 2017;
Vaswani et al., 2017, NMT;) models are data-hungry and their performances are
highly dependent upon the quantity and quality of labeled data, which are
expensive and scarce resources (Leong et al., 2021). This motivates the
research line of exploiting unlabeled monolingual data for boosting the model
performance of NMT. Due to simplicity and effectiveness, pre-training (Devlin
et al., 2019; Song et al., 2019, PT;) and back-translation (Sennrich et al.,
2016b, BT;) are two widely-used techniques for NMT, by leveraging a large
amount of monolingual data.

While empirically successful, the understandings of PT and BT are still
limited at best. Several attempts have been made to better understand them at
the data level, e.g. exploring different kinds of noises for the source data
(Edunov et al., 2018; Lewis et al., 2020). However, there are few
understandings at the model level that how PT and BT affect the internal
module (e.g. encoder and decoder) of NMT models. As recent studies start to
combine PT and BT for better model performance (Conneau and Lample, 2019; Liu
et al., 2020b; Ding et al., 2021c), there is a pressing need to broaden the
understandings of them.

To this end, we introduce two probing tasks to investigate the effects of PT
and BT on the encoder and decoder modules, respectively. We find that PT
mainly contributes to the encoder module while BT brings more benefits to the
decoder module. This provides a good explanation for the performance
improvement of simply combining PT and BT. Motivated by this finding, we
explore a better combination method by leveraging Tagged BT (Caswell et al.,
2019). Experiments conducted on the WMT16 English-Romanian and English-Russian
benchmarks show that PT can nicely co-work with BT, leading to state-of-the-
art model performances. Extensive analyses show that the tagging mechanism is
helpful for enhancing the complementarity between PT and BT by improving the
translation of source-original sentences and low-frequency words.

Our main contributions are as follows:

  * •

We design two probing tasks to investigate the impact of PT and BT on NMT
models.

  * •

We empirically demonstrate the complementarity between PT and BT.

  * •

We show that Tagged BT further improves the complementarity between PT and BT.

##  2 Preliminaries

###  2.1 Background

#### Pre-Training for NMT

Self-supervised PT (Devlin et al., 2019; Song et al., 2019), which can acquire
knowledge from unlabeled monolingual data, has shown its effectiveness in
improving the model performance of NMT, especially for those language pairs
with smaller parallel corpora (Conneau and Lample, 2019).

The first research line treats pre-trained models as external knowledge to
guidance NMT to learn better representations (Yang et al., 2020a; Zhu et al.,
2020) and predictions (Chen et al., 2020). These methods are effective but
costly since the NMT architecture needed to be elaborately designed. Another
research line is directly taking the weights of pre-trained models to
initialize NMT models, which is easy to use and advancing the state-of-the-art
(Rothe et al., 2020; Lewis et al., 2020). In this paper, we treat pre-trained
mBART (Liu et al., 2020b) as our testbed for parameter initialization, whose
benefits have been sufficiently validated (Tran et al., 2020; Tang et al.,
2020; Liu et al., 2021a) by multiple translation directions.

In general, previous studies focus on designing novel architectures (Song et
al., 2019) and artificial noises for source sentences (Lin et al., 2020; Yang
et al., 2020b) but are still unclear why pre-training can boost the model
performance of NMT, which is this paper aims to investigate.

#### Back-Translation for NMT

BT is an alternative to leverage monolingual data for NMT (Sennrich et al.,
2016b). It first trains a reversed NMT model for translating target-side
monolingual data into synthetic parallel data, and then complements them with
the original parallel data to train the desired NMT model. To improve BT,
previous works put attention to the importance of diversity and complexity in
synthetic data, showing that adding symbols (e.g., noises and tags) to the
back-translated source can help NMT distinguish the data from various sources
and learn better representations (Fadaee and Monz, 2018; Wang et al., 2019;
Edunov et al., 2018; Caswell et al., 2019; Marie et al., 2020). The claims and
understandings from these works are chiefly at the data-level rather than the
model-level.

There also exists some works that combine PT and BT to further boost the model
performance (Conneau et al., 2020; Song et al., 2019; Liu et al., 2020b).
However, the relation between BT and PT has not been fully studied. In this
paper, we take the first step to understand BT and PT at the model-level and
improve the complementarity between PT and BT.

###  2.2 Experimental Setup

#### Data

We conducted experiments on the WMT16 English-Romanian (En-Ro) and English-
Russia (En-Ru) translation tasks, which are widely-used benchmarks of data
augmentation methods for NMT. The training/validation/test sets of the En-Ro
include 612K/2K/2K sentence pairs, while those of En-Ru include 2M/3K/3K
pairs. Towards better reproducibility, we directly used the BT data provided
by Sennrich et al.
(2016a)111http://data.statmt.org/rsennrich/wmt16_backtranslations, consisting
of 2.3M synthetic data for the En-Ro and 2.0M data for the En-Ru. All the data
are tokenized and split into sub-words (Sennrich et al., 2016c) by the mBART
tokenizer (Liu et al., 2020b).

#### Setting

To make a fair comparison, all the model architectures and parameters are the
same as the pre-trained
mBART.cc25.222https://github.com/pytorch/fairseq/tree/master/examples/mbart
The NMT model augmented with PT directly uses the mBART weights for parameter
initialization, while the other models randomly initialize their parameters.
The training follows Liu et al. (2020b) except that we tuned the learning rate
within [3e-5,1e-3] and the dropout within [0.3,0.5] for the vanilla model and
BT models. We used the single model with the best validation perplexity for
testing. The length penalty is 1.0 and the beam size is 5. We used sacreBLEU
(Post, 2018) to calculate BLEU (Papineni et al., 2002) and TER (Snover et al.,
2006) scores with the specific tokenization (Liu et al., 2020b) for Romanian
and the default tokenization for Russian.

##  3 Understanding PT and BT

In this section, we aim to better understand the similarities and differences
between PT and BT on improving model performance. We design two probing tasks
to study the research question: Which module of NMT do PT and BT respectively
play a greater role in enhancing translation quality?

Enc. | Dec. | PT | BT
---|---|---|---
BLEU | $\Delta$ | BLEU | $\Delta$
N | N | 33.7 | - | 33.7 | -
N | Y | 33.5 | -0.2 | 37.8 | +4.1
Y | N | 36.9 | +3.2 | 35.8 | +2.1
Y | Y | 37.7 | +4.0 | 38.3 | +4.6
Table 1: The probing tasks of PT and BT. NMT models are trained and evaluated
on the WMT16 En-Ro benchmark. “Y” denotes the corresponding parameters are
activated when augmented with PT or BT, while “N” denotes the inactive
operation. PT and BT respectively contribute more to the NMT encoder and
decoder.

###  3.1 Effects of PT on NMT

Given a pre-trained model, it is common to use its part or all parameters to
initialize the downstream tasks. We design four NMT models, which differ from
the NMT components (Encoder vs. Decoder) with parameter initialization manners
(Random vs. Pre-trained). As shown in Table 1, the variants are: 1) NN is a
vanilla NMT model, of which parameters are randomly initialized; 2) NY means
that parameters of NMT encoder are randomly initialized while those of decoder
are initialized with pre-training; 3) YN is contrary to NY; and 4) YY
indicates that the whole NMT parameters are initialized with the pre-trained
model. After that, the NMT models are fine-tuned on the parallel data with the
same training strategy.

#### PT Mainly Contributes to Encoder

As seen, the YY initialization strategy significantly improves the vanilla NMT
model by +4.0 BLEU scores, which reconfirms the effectiveness of PT for
translation tasks Liu et al. (2020b). By comparing NY and YN, we find that the
pre-trained encoder can still help the NMT to achieve +3.2 BLEU improvements
while the pre-trained decoder can only perform on par with the vanilla model
(i.e. -0.2 BLEU). This demonstrates that PT mainly contributes to the encoder
part of NMT model, and this claim is consistent with the conclusion with other
pre-trained models. For instance, Rothe et al. (2020) show that the NMT
encoder initialization is superior to the decoder one when using pre-trained
weights of BERT. We hypothesize that the performance boost with PT mainly
comes from the better ability of source-side understanding, which is
significant to NMT such as on disambiguating word senses Tang et al. (2019).

###  3.2 Effects of BT on NMT

A vanilla NMT model is trained on the original bi-text and then fine-tuned on
the mixture of the original and synthetic (i.e. back-translated) data. We also
design four NMT models, which differ from which parts of parameters are
updated at the fine-tuning stage. As shown in Table 1, the variants are: 1) NN
is a vanilla NMT model only trained on the original data; 2) NY indicates that
parameters of the NMT encoder are fixed while those of decoder are updated
during model fine-tuning; 3) YN acts in an opposite way compared with NY; 4)
YY means that the whole NMT parameters are updated at the fine-tuning stage.

#### BT Mainly Contributes to Decoder

BT has been sufficiently validated to improve the performance of NMT models
(Edunov et al., 2018, 2020). By exploiting additional target sentences, the
NMT decoder can be enhanced to generate more fluent sentences in the target
language. In contrast, the synthetic source sentences contain noises, which
may be less useful for improving the ability of understanding. The results
verify our hypothesis: BT mainly improves the decoder module of NMT. As seen,
fine-tuning the whole NMT model (i.e. YY) with BT data can gain the best
performance (+4.6 BLEU than the vanilla model), which shows the effectiveness
of BT method. Surprisingly, only fine-tuning the decoder (i.e. NY) can perform
close to YY model (37.8 vs. 38.3 BLEU), which confirms our claims. Compared
with NY, the YN model obtains relatively fewer improvements (+4.1 vs. +2.1
BLEU), showing that BT brings more benefits to the decoder than the encoder.

##  4 Improving PT and BT

The answer of the research question in Section 3 is: PT and BT respectively
contribute more to the NMT encoder and decoder, demonstrating that they are
orthogonal and complementary to each other. This finding motivates us to
better combine these two individual techniques together for further improving
NMT models.

###  4.1 Experiments

As detailed in Section 2.2, we conducted experiments on two commonly-used
benchmarks En-Ro and En-Ru. Besides, we train the BT models from scratch
instead of fine-tuning in Section 3.2. As YY models (in Table 1) always
achieve best performances when augmented PT or BT, we update all parameters of
NMT models in next experiments.

Model | En-Ro | En-Ru
---|---|---
BLEU | TER | BLEU | TER
Existing Baselines
XLM-R (Conneau et al., 2020) | 35.6 | - | - | -
mRASP (Lin et al., 2020) | 37.6 | - | - | -
mBART (Liu et al., 2020b) | 37.7 | - | - | -
Our Implemented Systems
Vanilla NMT | 33.7 | 48.6 | 28.8 | 61.6
\+ PT | 37.7 | 45.0 | 31.6 | 58.5
\+ BT | 38.4 | 45.0 | 31.1 | 59.2
\+ BT + PT | 41.2 | 42.6 | 33.2 | 57.1
\+ Tagged BT | 38.6 | 44.9 | 31.2 | 59.3
\+ Tagged BT + PT | 41.6 | 42.1 | 33.6 | 56.5

Table 2: Translation quality on the En-Ro and En-Ru benchmarks. “+” means
incorporating PT and (Tagged) BT into NMT models.

The results are shown in Table 2. We use the vanilla model as our baselines,
which are trained on original datasets with random initialization. Besides, we
report results on existing PT models as our strong baselines, including XLM-R,
mRASP, mBART. As seen, PT can significantly improve the translation quality in
all cases compared with vanilla baselines (averagely +2.5 BLEU), which is
consistent with (or better than) existing PT models (37.7 vs. 35.6$\sim$37.7
BLEU). Furthermore, two BT methods333Tagged BT is to add a special token at
the beginning of each back-translated source sentence. (i.e. BT and Tagged BT)
perform closely, which improves the standard NMT models by +3.5/+3.7 BLEU
points on average. Simply combining them (+BT+PT) can further boost
performances for NMT models across different sizes of datasets, showing the
robustness and effectiveness of this approach. Encouragingly, the combination
of Tagged BT and PT performs better than the simple one, leading to state-of-
the-art performances on the two benchmarks. Similar tendencies are observed in
terms of the TER scores. The above results illustrate the better
complementarity between PT and Tagged BT on improving translation quality for
NMT models.

Model | All |  Src |  Tgt
---|---|---|---
Vanilla | 33.7 | 29.4 | 38.3
\+ PT | 37.7 | 33.8 | 42.0
\+ BT | 38.4 | 31.5 | 45.4
\+ BT + PT | 41.2 | 33.3 | 48.6
\+ Tagged BT | 38.6 | 31.9 | 45.6
\+ Tagged BT + PT | 41.6 | 34.8 | 48.7
Table 3: Translation quality of source-original and target-original sentences
on the En-Ro benchmark. “Src” and “Tgt” respectively denote the sub-testsets
of source-original and target-original while “All” means the whole testset.

###  4.2 Analysis

We conducted extensive analyses to better understand the improvement of our
approach. All results are reported on the En-Ro benchmark.

#### Effects of Sentence Type

Recent studies have shown that the evaluation of BT is sensitive to the
sentences types, thus we report BLEU scores on the subsets of source-original
(Src-Ori) and target-original (Tgt-Ori) datasets (Zhang and Toral, 2019; Liu
et al., 2021a; Wang et al., 2021).444Src-Ori denotes the testing data
originating in the source language, while Tgt-Ori denotes the data translating
from the target language. Generally speaking, the translation of Src-Ori is
more important than that of Tgt-Ori for practical NMT systems (Graham et al.,
2020), thus its performance should be taken seriously. As shown in Table 3, PT
performs better on Src-Ori than BT (33.8 vs. 31.9 BLEU) while BT achieves
higher scores on Tgt-Ori than PT (45.6 vs. 42.0 BLEU). Besides, simply
combining PT and BT can improve the translation quality on both Src-Ori and
Tgt-Ori sentences, but the improvement of Src-Ori is lower than only using PT.
By introducing tagged BT, the model can achieve better performance than the
simple one, especially on source-original sentences. Takeaway: 1) PT and BT
complementary in terms of originality of sentences; 2) Tagged BT can alleviate
the bias of translating Tgt-Ori sentences which is significant to practical
NMT systems.

Model | All | Low | High
---|---|---|---
Vanilla | 62.8 | 48.5 | 64.6
\+ PT | 65.8 | 58.2 | 66.7
\+ BT | 65.9 | 57.5 | 67.1
\+ BT + PT | 67.8 | 60.8 | 68.8
\+ Tagged BT | 66.1 | 57.5 | 67.3
\+ Tagged BT + PT | 68.3 | 61.8 | 69.1
Table 4: F-measure of word translation according to frequency on the En-Ro
benchmark. “Low” and “High” respectively denote the buckets of low- and high-
frequency words while “All” means the whole words in the test set. Simply
combining PT and BT improves the model performance, while adding tags to BT
data further improves

#### Effects of Word Frequency

Data augmentation is an effective way to improve the translation quality of
low-frequency words (Sennrich et al., 2016b). Thus, we compare the performance
of the models on translating different frequencies of words. Specifically, we
employed compare-mt (Neubig et al., 2019) to calculate the f-measure of
translating low- and high-frequency words ($<$50 vs. $\geq$50). As shown in
Table 4, PT improves more on translating low-frequency words (58.2 vs. 57.5
scores) while BT performs better on high-frequency words (67.3 vs. 66.7
scores). Furthermore, the combination of PT and tagged BT achieves the best
performance on both low- and high-frequency words, leading to an overall
improvement on the whole words. Similar phenomenons can be observed by
combining self-training and BT (Ding et al., 2021b). Takeaway: 1) PT and BT
complementary in terms of frequency of words; 2) Tagged BT are more
complementary to PT on lexical translation.

##  5 Conclusion and Future Works

This paper broadens the understandings of pre-training (PT) and back-
translation (BT). We propose two probing tasks to investigate the impact of PT
and BT on each NMT module and find that PT is more beneficial to the encoder
while BT mainly improves the decoder. Experimental results on the WMT16
English-Romanian and English-Russian benchmarks show that PT is nicely
complementary to BT. We also demonstrate that Tagged BT (i.e., adding tags to
BT data) can further improve the complementarity of translating source-
original sentences and low-frequency words.

In the future, we would like to apply curriculum learning (Liu et al., 2020a;
Zhan et al., 2021; Ding et al., 2021a) to better organize the learning of PT
and BT. It is also worthwhile to explore other kinds of methods utilizing
monolingual data (e.g., self-training (Zhang and Zong, 2016; He et al., 2020;
Jiao et al., 2021)) and validate the findings on practical NMT systems (Huang
et al., 2021) and more generation tasks (Liu et al., 2021b).

## Acknowledgement

This work was supported in part by the Science and Technology Development
Fund, Macau SAR (Grant No. 0101/2019/A2), and the Multi-year Research Grant
from the University of Macau (Grant No. MYRG2020-00054-FST). We thank the
anonymous reviewers for their insightful comments.

## References

  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Caswell et al. (2019) Isaac Caswell, Ciprian Chelba, and David Grangier. 2019.  Tagged back-translation.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)_ , pages 53–63, Florence, Italy. Association for Computational Linguistics.
  * Chen et al. (2020) Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, and Jingjing Liu. 2020.  Distilling knowledge learned in BERT for text generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7893–7905, Online. Association for Computational Linguistics.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Conneau and Lample (2019) Alexis Conneau and Guillaume Lample. 2019.  Cross-lingual language model pretraining.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 7057–7067.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Ding et al. (2021a) Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, and Zhaopeng Tu. 2021a.  Progressive multi-granularity training for non-autoregressive translation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 2797–2803, Online. Association for Computational Linguistics.
  * Ding et al. (2021b) Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, and Zhaopeng Tu. 2021b.  Rejuvenating low-frequency words: Making the most of parallel data in non-autoregressive translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3431–3441, Online. Association for Computational Linguistics.
  * Ding et al. (2021c) Liang Ding, Di Wu, and Dacheng Tao. 2021c.  Improving neural machine translation by bidirectional training.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics.
  * Edunov et al. (2018) Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018.  Understanding back-translation at scale.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 489–500, Brussels, Belgium. Association for Computational Linguistics.
  * Edunov et al. (2020) Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, and Michael Auli. 2020.  On the evaluation of machine translation systems trained with back-translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2836–2846, Online. Association for Computational Linguistics.
  * Fadaee and Monz (2018) Marzieh Fadaee and Christof Monz. 2018.  Back-translation sampling by targeting difficult words in neural machine translation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 436–446, Brussels, Belgium. Association for Computational Linguistics.
  * Gehring et al. (2017) Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017\.  Convolutional sequence to sequence learning.  In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 1243–1252. PMLR.
  * Graham et al. (2020) Yvette Graham, Barry Haddow, and Philipp Koehn. 2020.  Statistical power and translationese in machine translation evaluation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 72–81, Online. Association for Computational Linguistics.
  * He et al. (2020) Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2020.  Revisiting self-training for neural sequence generation.  In _International Conference on Learning Representations_.
  * Huang et al. (2021) Guoping Huang, Lemao Liu, Xing Wang, Longyue Wang, Huayang Li, Zhaopeng Tu, Chengyan Huang, and Shuming Shi. 2021.  Transmart: A practical interactive machine translation system.  _CoRR_ , abs/2105.13072.
  * Jiao et al. (2021) Wenxiang Jiao, Xing Wang, Zhaopeng Tu, Shuming Shi, Michael Lyu, and Irwin King. 2021.  Self-training sampling with monolingual data uncertainty for neural machine translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 2840–2850, Online. Association for Computational Linguistics.
  * Leong et al. (2021) Chongman Leong, Xuebo Liu, Derek F. Wong, and Lidia S. Chao. 2021.  Exploiting translation model for parallel corpus mining.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 29:2829–2839.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lin et al. (2020) Zehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu, Jiangtao Feng, Hao Zhou, and Lei Li. 2020.  Pre-training multilingual neural machine translation by leveraging alignment information.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2649–2663, Online. Association for Computational Linguistics.
  * Liu et al. (2020a) Xuebo Liu, Houtim Lai, Derek F. Wong, and Lidia S. Chao. 2020a.  Norm-based curriculum learning for neural machine translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 427–436, Online. Association for Computational Linguistics.
  * Liu et al. (2021a) Xuebo Liu, Longyue Wang, Derek F. Wong, Liang Ding, Lidia S. Chao, Shuming Shi, and Zhaopeng Tu. 2021a.  On the copying behaviors of pre-training for neural machine translation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4265–4275, Online. Association for Computational Linguistics.
  * Liu et al. (2021b) Xuebo Liu, Longyue Wang, Derek F. Wong, Liang Ding, Lidia S. Chao, and Zhaopeng Tu. 2021b.  Understanding and improving encoder layer fusion in sequence-to-sequence learning.  In _International Conference on Learning Representations_.
  * Liu et al. (2020b) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020b.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Marie et al. (2020) Benjamin Marie, Raphael Rubino, and Atsushi Fujita. 2020.  Tagged back-translation revisited: Why does it really work? In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5990–5997, Online. Association for Computational Linguistics.
  * Neubig et al. (2019) Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, and Xinyi Wang. 2019.  compare-mt: A tool for holistic comparison of language generation systems.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 35–41, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  * Rothe et al. (2020) Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020.  Leveraging pre-trained checkpoints for sequence generation tasks.  _Transactions of the Association for Computational Linguistics_ , 8:264–280.
  * Sennrich et al. (2016a) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a.  Edinburgh neural machine translation systems for WMT 16.  In _Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers_ , pages 371–376, Berlin, Germany. Association for Computational Linguistics.
  * Sennrich et al. (2016b) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b.  Improving neural machine translation models with monolingual data.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 86–96, Berlin, Germany. Association for Computational Linguistics.
  * Sennrich et al. (2016c) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016c.  Neural machine translation of rare words with subword units.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
  * Snover et al. (2006) Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006.  A study of translation edit rate with targeted human annotation.  In _Proceedings of association for machine translation in the Americas_ , volume 200. Citeseer.
  * Song et al. (2019) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019.  MASS: masked sequence to sequence pre-training for language generation.  In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 5926–5936. PMLR.
  * Tang et al. (2019) Gongbo Tang, Rico Sennrich, and Joakim Nivre. 2019.  Encoders help you disambiguate word senses in neural machine translation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1429–1435, Hong Kong, China. Association for Computational Linguistics.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.  _arXiv preprint arXiv:2008.00401_.
  * Tran et al. (2020) Chau Tran, Yuqing Tang, Xian Li, and Jiatao Gu. 2020.  Cross-lingual retrieval for iterative self-supervised training.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 2207–2219. Curran Associates, Inc.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Wang et al. (2019) Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, and Maosong Sun. 2019.  Improving back-translation with uncertainty-based confidence estimation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 791–802, Hong Kong, China. Association for Computational Linguistics.
  * Wang et al. (2021) Shuo Wang, Zhaopeng Tu, Zhixing Tan, Shuming Shi, Maosong Sun, and Yang Liu. 2021\.  On the language coverage bias for neural machine translation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4778–4790, Online. Association for Computational Linguistics.
  * Yang et al. (2020a) Jiacheng Yang, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Weinan Zhang, Yong Yu, and Lei Li. 2020a.  Towards making the most of bert in neural machine translation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 9378–9385.
  * Yang et al. (2020b) Zhen Yang, Bojie Hu, Ambyera Han, Shen Huang, and Qi Ju. 2020b.  CSP:code-switching pre-training for neural machine translation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2624–2636, Online. Association for Computational Linguistics.
  * Zhan et al. (2021) Runzhe Zhan, Xuebo Liu, Derek F. Wong, and Lidia S. Chao. 2021.  Meta-curriculum learning for domain adaptation in neural machine translation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 14310–14318.
  * Zhang and Zong (2016) Jiajun Zhang and Chengqing Zong. 2016.  Exploiting source-side monolingual data in neural machine translation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1535–1545, Austin, Texas. Association for Computational Linguistics.
  * Zhang and Toral (2019) Mike Zhang and Antonio Toral. 2019.  The effect of translationese in machine translation test sets.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)_ , pages 73–81, Florence, Italy. Association for Computational Linguistics.
  * Zhu et al. (2020) Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. 2020.  Incorporating BERT into neural machine translation.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.


# Optimal Narain CFTs from Codes

Nikolaos Angelinos, Debarghya Chakraborty, Anatoly Dymarsky
Department of Physics and Astronomy,
University of Kentucky,
Lexington, KY, USA 40506

###### Abstract:

Recently established connection between additive codes and Narain CFTs
provides a new tool to construct theories with special properties and solve
modular bootstrap constraints by reducing them to algebraic identities. We
generalize previous constructions to include many new theories, in particular
we show that all known optimal Narain theories, i.e. those maximizing the
value of spectral gap, can be constructed from codes. For asymptotically large
central charge $c$ we show there are code theories with the spectral gap
growing linearly with $c$, with the coefficient saturating the conjectural
upper bound. We therefore conjecture that optimal Narain theories for any
value of $c$ can be obtained from codes.

##  1 Introduction

Conformal modular bootstrap program aims to establish universal constraints on
two-dimensional CFTs and elucidate properties of those special theories which
saturate these constraints. One of the central goals of the modular bootstrap
is to study theories maximizing the value of spectral gap for given fixed
value of central charge [1], as these theories for large central charge are
expected to be dual to weakly coupled gravity [2]. To simplify this obviously
challenging task one can restrict attention to a class of Narain theories,
i.e. CFTs exhibiting $U(1)^{c}\times U(1)^{c}$ symmetry. In this case large
spectral gap theories are not sparse (in the sense of [2]), and their
holographic description is less clear [3]. Nevertheless study of such theories
is well motivated by both holography and the modular bootstrap, with the
latter relating solutions of spinless bootstrap constraints to densest sphere
packings [4].111Here we are speaking of a density of states satisfying (some
subset) of modular bootstrap constraints, with no regard to whether there is
an actual CFT yielding this density of states. Similarly, speaking of densest
sphere packings, we in fact refer to a solution to Cohen-Elkies linear program
constraints [5], with no regard to whether there are actual associated sphere
packings.  Narain theories were studied in [6] and [7] using spinless and full
modular bootstrap, with the hypothetical optimal theories being identified for
$c\leq 8$. Here, following [8] we say optimal to denote CFTs maximizing
spectral gap for given $c$.

A relation between quantum codes and Narain CFTs, proposed in [9], generalizes
chiral constructions of [10]. Starting from a code it constructs corresponding
Narain lattice and expresses CFT torus partition function in terms of the code
enumerator polynomial (for higher genus generalization see [11, 12]). In this
way constraints of modular invariance reduce to two algebraic constrains at
the level of enumerator polynomial. The relation to quantum codes was recently
extended and interpreted in terms of CFT Hilbert space in [13]. There are also
“bottom-up” generalizations when the connection with codes is perceived as a
tool to solve modular bootstrap constraints and construct interesting CFTs
[14, 15], also see [16, 17] for the subsequent developments. In this paper we
use this approach and introduce the umbrella construction which generalizes
and encompasses the constructions of [14, 15]. In particular we show that all
(conjecturally) optimal Narain theories for $c\leq 8$ identified in [7] are in
fact code theories, by providing an explicit way to construct these theories
from codes.

Our construction should be understood as an infinite family of similar but
distinct constructions. We consider codes over abelian groups ${\sf
G}=\mathbb{Z}_{p}\times\mathbb{Z}_{q}$ equipped with particular scalar product
and map even self-dual codes ${\mathcal{C}}\subset{\sf G}^{c}$ to Narain
lattices using a suitable generalization of the Construction A of [18]. The
same group ${\sf G}$ might be mapped into lattices in several different ways,
each way defining a particular construction. Then an appropriate
generalization of the code Hamming distance (we explain what this is latter in
the text), modulo certain subtleties, defines CFT spectral gap $\Delta^{*}$
such that “better” codes with larger Hamming distance corresponds to larger
$\Delta^{*}$. In each case, the resulting Narain lattice necessarily has
vectors of particular length which is independent of $c$. Hence any given
construction can only yield CFTs with bounded spectral gap that doesn’t grow
with $c$. Nevertheless by considering a sequence of constructions parametrized
by $c$ one can obtain a family of Narain theories with the spectral gap
growing linearly with $c$. For $c\gg 1$ finding optimal codes, i.e. those
maximizing corresponding Hamming distance, is a challenging task, but one can
average over a family of codes with the given $c$. From here we find that
random code CFT, drawn from a particular ensemble, has spectral gap

$\displaystyle\Delta^{*}={c\over 2\pi e},\qquad c\rightarrow\infty,$ (1.1)

which was conjectured in [3] to be asymptotically largest possible value.
Thus, we conclude that certain code CFT are optimal for $c\gg 1$ or at least
give spectral gap with the conjectured maximal asymptotic value of
$\Delta^{*}/c$.

The paper is organized as follows. In section 2 we outline our main
construction mapping codes to Narain CFTs and then express their partition
functions in terms of enumerator polynomials in section 3. We then use these
results to construct optimal theories for $c\leq 8$ in section 4. We proceed
by considering the case of asymptotically large $c$ and a family of associated
constructions in section 5. We conclude in section 6.

Note: recently a paper [16] appeared which has an overlap with the square
lattice construction discussed in sections 2.3, 3.2.

##  2 Additive codes and Lorentzian lattices

The main ingredient of our construction is a 2-dimensional even lattice
$\Lambda\in\mathbb{R}^{1,1}$, which we call a glue lattice. Starting from such
a lattice, we define an additive group ${\sf G}$, which serves as the alphabet
of the code. Then standard Construction A maps a code
${\mathcal{C}}\subset{\sf G}^{c}$ into a lattice

$\displaystyle\underbrace{\Lambda\oplus\dots\oplus\Lambda}_{c\,{\rm
times}}\subset\Lambda_{\mathcal{C}}\subset\underbrace{\Lambda^{\perp}\oplus\dots\Lambda^{\perp}}_{c\,{\rm
times}}\subset\mathbb{R}^{c,c}.$ (2.2)

When ${\mathcal{C}}$ satisfies additional conditions, the lattice
$\Lambda_{\mathcal{C}}$ is even and self-dual, thus defining a Narain theory.

###  2.1 Even lattices in $\mathbb{R}^{1,1}$

We equip $\mathbb{R}^{2}$ with a Lorentzian metric

$g=\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},$ (2.3)

thus turning it into $\mathbb{R}^{1,1}$. A two-dimensional “glue” lattice
$\Lambda\subset\mathbb{R}^{1,1}$ is called integral if $v^{T}gu\in\mathbb{Z}$
for any $u,v\in\Lambda$. A lattice called even if $v^{T}gv\in 2\mathbb{Z}$ for
any $v\in\Lambda$. Any even lattice is automatically integral.

It is convenient to parametrize a lattice by a generating matrix $\Lambda$,
such that $v=\Lambda n$, $n\in\mathbb{Z}^{2}$ generates all lattice vectors.
Here we abuse the notations by using $\Lambda$ to denote both the lattice and
its generating matrix. The generating matrix is not unique, obviously
$\Lambda$ and $\Lambda S$ for $S\in SL(2,\mathbb{Z})$ generate the same
lattice. So far we are only interested in the scalar product defined by (2.3),
we can identify all lattices related by $O(1,1)$, $\Lambda\sim O\Lambda$ for
$O\in O(1,1)$.

By dual lattice $\Lambda^{\perp}$ we understand all vectors
$v\in\Lambda^{\perp}\subset\mathbb{R}^{1,1}$ such that $v^{T}gu\in\mathbb{Z}$
for any vector $u\in\Lambda$. At the level of generating matrix
$\Lambda^{\perp}=g(\Lambda^{-1})^{T}$.

We can parametrize all even lattices in $\mathbb{R}^{1,1}$ as follows. In full
generality

$g_{\Lambda}=\Lambda^{T}g\Lambda=\begin{pmatrix}2m&k\\\
k&2n\end{pmatrix},~{}n,m,k\in\mathbb{Z},$ (2.4)

assuming $\det(\Lambda)=\sqrt{k^{2}-4mn}>0$. Then using $SL(2,\mathbb{Z})$ we
can bring $n,m,k$ to satisfy (see chapter 15 of [18])

$\displaystyle 0<k<\sqrt{k^{2}-4nm}<\min(k+2|n|,k+2|m|),$ (2.5)

unless $k^{2}-4nm$ is a full square, in which case one can choose new $n,m,k$
such that

$\displaystyle n=0,\quad-k<m\leq k.$ (2.6)

With the help of an appropriate $O(1,1)$ transformation the corresponding
generating matrix can be brought to the form

$\displaystyle\Lambda=\begin{pmatrix}1&{n\over a}\\\
m&a\end{pmatrix},~{}a={1\over 2}(k+\sqrt{k^{2}-4mn}).$ (2.7)

Since $\Lambda$ is integral, it is contained in its dual
$\Lambda^{\perp}\supseteq\Lambda$. We call the following quotient the “glue
group”

${\sf
G}\>=\Lambda^{\perp}/\Lambda=\mathbb{Z}^{2}/g_{\Lambda}=\mathbb{Z}_{p}\times\mathbb{Z}_{q}.$
(2.8)

Here

$\displaystyle p=gcd(2n,2m,k),\qquad q=|{\sf G}|/p,$ (2.9)

where $|{\sf G}|$ is the order of the group. This follows from the invariant
factor decomposition of finitely generated abelian groups.

$|{\sf G}|=|\det(\Lambda)/\det(\Lambda^{\perp})|=k^{2}-4mn.$ (2.10)

One possible parametrization of the elements of ${\sf G}$ is as pairs ${\sf
g}=(a,b)$ of integer numbers $0\leq a<p,\,0\leq b<q$. Another useful
parametrization is as integer vectors $\ell({\sf g})\in\mathbb{Z}^{2}$ modulo
columns of $g_{\Lambda}$. An explicit map between these two parametrizations
may be nontrivial.

The metric on $\mathbb{R}^{1,1}$ induces a scalar product on ${\sf G}$, which
we denote by $\eta$. It is defined up to shifts by integer numbers, which
reflects different choices of representatives in the quotient
$\Lambda^{\perp}/\Lambda$. One can consider codes over ${\sf G}$, namely
additive linear subspaces ${\mathcal{C}}\subset{\sf G}^{c}$. A code would be
called even if the norm of each element defined with $\eta$ is even

$\displaystyle\eta({\sf c},{\sf c})=\sum_{i=1}^{c}\eta({\sf g}_{i},{\sf
g}_{i})=\sum_{i=1}^{c}\ell({\sf g}_{i})^{T}g_{\Lambda}^{-1}\ell({\sf
g_{i}})\in 2\mathbb{Z},\quad{\sf c}\equiv({\sf g}_{1},\dots,{\sf
g}_{c})\in{\mathcal{C}}.$ (2.11)

Despite that $\eta$ is defined only up to certain integer shifts, whether
$\eta({\sf g},{\sf g})$ is even does not depend on the representative of $\sf
g$. With help of $\eta$ one can also define dual code ${\mathcal{C}}^{\perp}$,
as the group of all elements ${\sf c}_{1}\in{\mathcal{C}}^{\perp}\subset{\sf
G}^{c}$ satisfying $\eta({\sf c}_{1},{\sf c}_{2})\in\mathbb{Z}$ for any ${\sf
c}_{2}\in{\mathcal{C}}$. A code is called self-dual when
${\mathcal{C}}={\mathcal{C}}^{\perp}$.

###  2.2 Construction A

Starting from a code ${\mathcal{C}}\subset{\sf G}^{c}$, Construction A
associates to it the lattice $\Lambda_{\mathcal{C}}$,

$\displaystyle\underbrace{\Lambda\oplus\dots\oplus\Lambda}_{c\,{\rm
times}}\subset\Lambda_{\mathcal{C}}\subset\underbrace{\Lambda^{\perp}\oplus\dots\oplus\Lambda^{\perp}}_{c\,{\rm
times}}\subset\mathbb{R}^{c,c},$ (2.12)

defined as the set of vectors in
$\Lambda^{\perp}\oplus\dots\oplus\Lambda^{\perp}$ mapped into
${\mathcal{C}}\subset{\sf G}^{c}$ under (2.8). Then it is straightforward to
see that an even ${\mathcal{C}}$ would give rise to an even lattice
$\Lambda_{\mathcal{C}}$ and a self-dual ${\mathcal{C}}$ to a self-dual
$\Lambda_{\mathcal{C}}$, both understood with respect to Lorentzian scalar
product $g_{L}=g\oplus\dots\oplus g$ in $\mathbb{R}^{c,c}$.

To define a Narain theory, besides Lorentzian scalar product, even self-dual
lattice should also be equipped with the Euclidean scalar product. For each
$\Lambda$ defined in previous section there is $O(1,1)$ ambiguity how it can
be embedded in $\mathbb{R}^{2}$. Thus, very explicitly we can write

$\displaystyle v=\begin{pmatrix}O(\Lambda^{\perp}\ell_{1}+\Lambda k_{1})\\\
\vdots\\\ O(\Lambda^{\perp}\ell_{c}+\Lambda k_{c})\\\
\end{pmatrix}\in\Lambda_{\mathcal{C}},\quad(\ell_{1},\dots,\ell_{c})\in{\mathcal{C}},\quad
k_{i}\in\mathbb{Z}^{2},$ (2.13)

where in (2.13) we parametrize elements of ${\sf
G}=\mathbb{Z}^{2}/g_{\Lambda}$ by vectors $\ell$. Matrix $O$ is an arbitrary
element from $O(1,1)$. In principle we can introduce $c$ different
transformations $O_{i}\in O(1,1)$ acting in each $\mathbb{R}^{2}$ plane. In
this case most of the construction remains intact, but the permutation of
factors of ${\sf G}$ in ${\mathcal{C}}\subset{\sf G}^{c}$, which is
conventionally considered to be a code equivalence, would no longer yield
physically equivalent lattices. In what follows we assume that all factors $O$
are the same.

The main result of this section is as follows. For any glue group ${\sf G}$
defined via (2.8) with the help of an appropriate even lattice
$\Lambda\subset\mathbb{R}^{1,1}$, any even self-dual code
${\mathcal{C}}\subset{\sf G}^{c}$ via Construction A defines Narain lattice
$\Lambda_{\mathcal{C}}$ (2.13) and hence a Narain CFT. We will call such CFTs
code theories.

###  2.3 Example: square glue lattice

Consider the following glue lattice generating matrix

$\Lambda=\sqrt{p}\,g,\quad p\in\mathbb{N}.$ (2.14)

The dual lattice is generated by $\Lambda^{\perp}=I/\sqrt{p}$. Clearly this is
the case of $n=m=0,k=p$ and the glue group ${\sf
G}=\mathbb{Z}_{p}\times\mathbb{Z}_{p}$ is parametrized by ${\sf
g}=(a,b)\in{\sf G}$, $0\leq a,b<p$ and

$\displaystyle\ell({\sf g})=(a,b)^{T}.$ (2.15)

It is convenient to write codewords ${\sf c}=({\sf g_{1}},\dots,{\sf
g}_{c})\in{\mathcal{C}}\subset{\sf G}^{c}$ as

$\displaystyle{\sf
c}=(a_{1},\dots,a_{c}|b_{1},\dots,b_{c})\in\mathbb{Z}_{p}^{2c}.$ (2.16)

A code can be defined with a $2c\times d$ generating matrix $G_{\mathcal{C}}$
such that

$\displaystyle{\sf c}=G_{\mathcal{C}}\,r,\quad r\in\mathbb{Z}_{p}^{d},$ (2.17)

where $d$ depends on $p$. For prime $p$ generating matrix is $2c\times c$ and
using permutations can always be brought to the form

$\displaystyle G_{\mathcal{C}}=\left(I\,|\,B^{T}\right),$ (2.18)

where $B$ is an integer values antisymmetric $c\times c$ matrix defined mod
$p$,

$\displaystyle B^{T}=-B\mod\,p,$ (2.19)

and $B_{ii}=0$. Applying Construction A to such codes leads to Narain lattices
generated by

$\displaystyle\Lambda_{\mathcal{C}}=\left(\begin{array}[]{cc}pI_{c}&B\\\
0&I_{c}\end{array}\right)/\sqrt{p},\qquad
g_{L}=\left(\begin{array}[]{cc}0&I_{c}\\\ I_{c}&0\end{array}\right),$ (2.24)

associated with the Lorentzian scalar product $g_{L}$. Here $I_{c}$ is a
$c$-dimensional identity matrix.

The form of $\Lambda_{\mathcal{C}}$ provides a clear interpretation –
corresponding Narain theory describes $c$ scalars compactified on a
$c$-dimensional cube of size $1/\sqrt{p}$ in presence of B-field $B$.

The construction described above is a straightforward generalization of the
original construction of [14, 9], which considered the case of $p=2$. It has
been recently introduced independently in [16].

###  2.4 Generalization: isodual codes

Permutation of factors $S:{\sf G}^{c}\rightarrow{\sf G}^{c}$

$\displaystyle S:({\sf g}_{1},\dots,{\sf g}_{c})\rightarrow({\sf
g}_{i_{1}},\dots,{\sf g}_{i_{c}})$ (2.25)

is the simplest example of code equivalences, defined as a linear
transformation $S:{\sf G}^{c}\rightarrow{\sf G}^{c}$ which preserves scalar
product $\eta$. Provided dual code is equivalent to the original one

$\displaystyle{\mathcal{C}}^{\perp}=S({\mathcal{C}}),$ (2.26)

such a code is called isodual. From this follows $S^{2}=1$, i.e. when $S$ is a
permutation, it is a pair-wise permutation, with the corresponding matrix
satisfying $S^{T}=S$.

We can introduce evenness as the condition for all codeword $({\sf
g}_{1},\dots,{\sf g}_{c})\in{\mathcal{C}}\subset{\sf G}^{c}$ to have even
scalar product with its permuted self,

$\displaystyle\sum_{i=1}^{c}\eta({\sf g}_{i},{\sf g}_{S(i)})\in 2\mathbb{Z}.$
(2.27)

An even, isodual code ${\mathcal{C}}$ with respect to some pairwise
permutation $S$, via Construction A (2.13) gives rise to an even lattice,
which is self-dual with respect to Lorentzian scalar product

$\displaystyle g_{L}=g\otimes S.$ (2.28)

In this way isodual codes also can be used to define Narain lattices and code
CFTs.

###  2.5 CFT spectral gap and code modified Hamming distance

The capacity of a code to preserve information is rooted in the ability to
unambiguously restore ${\sf c}\in{\mathcal{C}}$ in case it got corrupted
during transmission process. Speaking colloquially, the code is better
(stronger), provided all codewords are maximally distinct from each other. To
quantify that coding theory uses Hamming distance $d({\sf c},{\sf
c}^{\prime})$, defined as the number of different components of ${\sf c}$,
${\sf c}^{\prime}$. For the binary linear code, when the components of ${\sf
c}$ are either zero or one, $d({\sf c},{\sf c}^{\prime})=|{\sf c}-{\sf
c}^{\prime}|_{1}=|{\sf c}-{\sf c}^{\prime}|_{2}^{2}$. When the code is not
binary, there could be various generalizations of Hamming distance, relevant
for different practical scenarios. Thus, if the transmission error changes a
codeword component’s value randomly, with the new value being unrelated to the
original one, then Hamming distance defined as the number of distinct
components is still relevant. Yet, in many practical settings the nature of
errors are such that the new values tend to be closer to old ones, which
suggest using various metrics. In particular in case of codes over
$\mathbb{Z}_{p}$ one often defines

$\displaystyle d({\sf c},{\sf c}^{\prime})=|{\sf c}-{\sf
c}^{\prime}|_{2}^{2}=\sum_{i=1}^{c}|a_{i}-a^{\prime}_{i}|^{2},$ (2.29)
$\displaystyle{\sf
c}=(a_{1},\dots,a_{c})\in{\mathcal{C}}\subset\mathbb{Z}_{p}^{c},\quad 0\leq
a_{i}<p.$ (2.30)

For a linear code, the code distance is

$\displaystyle d({\mathcal{C}})=\min_{0\leq i,j\leq|{\mathcal{C}}|-1}d({\sf
c}_{i},{\sf c}_{j})=\min_{{\sf c}\in{\mathcal{C}},{\sf c}\neq 0}d(0,{\sf c}).$
(2.31)

The value $w({\sf c}):=d(0,{\sf c})$ is usually called the weight of ${\sf
c}$.

Construction A (2.13) relates each codeword ${\sf c}\in{\mathcal{C}}$ to a
family of vectors $v({\sf c})$. We define the weight of ${\sf c}$ to be the
minimal length square among all such $v$,

$\displaystyle w({\sf c})$ $\displaystyle=$
$\displaystyle\min_{k_{1},\dots,k_{c}\in\mathbb{Z}}|v|^{2},$ (2.32)
$\displaystyle v$ $\displaystyle=$ $\displaystyle(\ell_{1}+\Lambda
k_{1},\dots,\ell_{c}+\Lambda k_{c}),\qquad{\sf
c}=(\ell_{1},\dots,\ell_{c})\in{\mathcal{C}}\subset{\sf G}^{c},\quad{\sf
G}=\mathbb{Z}^{2}/g_{\Lambda}.$

The weight $w({\sf d})$ is closely related to the norm of ${\sf c}$ calculated
with the scalar product $\eta$ inherited from $\mathbb{R}^{1,1}$ on ${\sf G}$.

Clearly, from this definition follows that code theory spectral gap
$\Delta^{*}$, defined as the length-squared of the shortest non-zero vector
divided by two, is simply related to code’s generalized Hamming distance
$\Delta^{*}={d({\mathcal{C}})\over 2}.$ This relation is transparent, but
there is one caveat: zero codeword ${\sf c}=0$ is mapped into the origin of
$\Lambda_{\mathcal{C}}$, as well as many vectors of the form
$\Lambda(k_{1},\dots,k_{c})$. The origin is excluded from the consideration,
while minimizing over $k_{i}$ yields shortest vector of $\Lambda$. We thus
have

$\displaystyle\Delta^{*}={1\over
2}\min\left({d({\mathcal{C}})},{|v_{\Lambda}|^{2}}\right),$ (2.33)

where by $|v_{\Lambda}|$ we denoted the length of shortest non-trivial vector
of $\Lambda$. This length depends non-trivially on the choice of $n,m,k$ and
the $O(1,1)$ factor (which we absorbed into the definition of $\Lambda$), but
an upper bound (2.36) is readily available, see below.

For the square lattice of subsection 2.3 we find that $d({\sf c},{\sf
c^{\prime}})$ is given by (2.29) where by $|a-a^{\prime}|^{2}$ with $0\leq
a,a^{\prime}<p$ we understand

$\displaystyle|a-a^{\prime}|^{2}:=\min_{k\in Z}\,(a-a^{\prime}+pk)^{2},$
(2.34)

and $|v_{\Lambda}|^{2}=p$.

To obtain the upper bound on the length, in the sense of Euclidean norm, of
the shortest vector $v_{\Lambda}$ belonging to (2.13) we consider all
$\ell_{i}=0$, arbitrary $k_{1}$ and $k_{i}=0$ for $i>1$. Then the Euclidean
norm of corresponding two-dimensional vectors is

$\displaystyle|v|^{2}=k_{1}^{T}\Lambda^{T}O^{T}O\Lambda k_{1},$ (2.35)

which defines a positive-definite scalar product in $\mathbb{R}^{2}$. The
shortest vector will necessarily be shorter than $|v_{\Lambda}|^{2}\leq 2|{\sf
G}|^{1/2}/\sqrt{3}$, see Appendix A, and therefore corresponding code CFT
would have the spectral gap not exceeding

$\displaystyle\Delta^{*}\leq{\sqrt{k^{2}-4nm}\over\sqrt{3}}.$ (2.36)

This is a standard weakness of the Construction A lattices; they always
include short vectors of a certain length, which does not increase with $c$.
Therefore to construct large spectral gap CFTs with $\Delta^{*}$ scaling
linearly with $c$ we would need to consider a cascade of different
constructions by adjusting $k,n,m$ together with $c$.

##  3 Torus partition function of code theories

###  3.1 Enumerator polynomial and theta-series

One of the central properties of code theories which make them interesting is
that their torus partition function can be expressed in a compact way in terms
of the so-called code enumerator polynomial which characterizes the
corresponding code. Generalization of this result to higher genus partition
function is also possible [19, 20].

We first define the full enumerator polynomial of a code ${\mathcal{C}}$ as a
vehicle to count how many times each element ${\sf g}\in{\sf G}$ appears in
each codeword of ${\mathcal{C}}$,

$P_{C}(\\{x_{\sf g}\\})=\sum_{({\sf g}_{1},\dots,{\sf
g}_{c})\in{\mathcal{C}}}\prod_{i=1}^{c}x_{{\sf g}_{i}}.$ (3.37)

This is a degree $n$ homogeneous polynomial of $|{\sf G}|$ variables $x_{\sf
g}$.

The polynomial of the dual code ${\mathcal{C}}^{\perp}$ is related to the
polynomial of ${\mathcal{C}}$ by the MacWilliams identity

$P_{{\mathcal{C}}^{\perp}}(\\{\tilde{x}_{\sf g}\\})=P_{\mathcal{C}}(\\{x_{\sf
g}\\}),$ (3.38)

where

$\tilde{x}_{\tilde{\sf g}}={1\over\sqrt{\left|{\sf G}\right|}}\sum_{{\sf
g}\in{\sf G}}\exp(-2\pi i\eta(\tilde{\sf g},{\sf g}))x_{\sf g}.$ (3.39)

In particular, this means that the enumerator polynomial of a self-dual code
is invariant under the above transformation.

The torus partition function of a code CFT, associated with
$\Lambda_{\mathcal{C}}$ obtained by Construction A, is given in terms of
$P_{\mathcal{C}}$,

$\mathcal{Z}_{{\mathcal{C}}}(\tau)={P_{\mathcal{C}}(\\{\psi_{\sf
g}(\tau)\\})},$ (3.40)

where

$\displaystyle\psi_{\ell}(\tau)={1\over|\eta(\tau)|^{2}}\sum_{k\in\mathbb{Z}^{2}}\exp(i\pi
v^{T}\Omega v),$ (3.41) $\displaystyle
v=\Lambda^{\perp}\ell+\Lambda\,k,\quad\Omega=\begin{pmatrix}i\tau_{2}&\tau_{1}\\\
\tau_{1}&i\tau_{2}\end{pmatrix},$ (3.42)

$\tau=\tau_{1}+i\tau_{2}$ is the torus modular parameter and in (3.41) we
parametrize elements of ${\sf G}$ with help of vectors
$\ell\in\mathbb{Z}^{2}/g_{\Lambda}$. We also absorbed $O\in O(1,1)$ into the
definition of $\Lambda$.

The modular group, generated by $T:\tau\to\tau+1$ and $S:\tau\to-1/\tau$,
transforms $\Omega$ as follows

$T\circ\Omega=\Omega+g,$ (3.43) $S\circ\Omega=-\Omega^{-1}.$ (3.44)

Functions $\psi_{\ell}$ transform accordingly

$\displaystyle T\circ\psi_{\ell}(\tau)$ $\displaystyle=$
$\displaystyle\psi_{\ell}(\tau+1)=\exp(i\pi
v^{T}g_{\Lambda}v)\psi_{\ell}(\tau),\quad v=\Lambda^{\perp}\ell,$ (3.45)
$\displaystyle S\circ\psi_{\ell}(\tau)$ $\displaystyle=$
$\displaystyle\psi_{\ell}(-1/\tau)={1\over\sqrt{\left|{\sf
G}\right|}}\sum_{\ell^{\prime}\in{\sf G}=\mathbb{Z}^{2}/g_{\Lambda}}\exp(-2\pi
iu^{T}g_{\Lambda}v)\psi_{\ell^{\prime}}(\tau),\quad
u=\Lambda^{\perp}\ell^{\prime}.\qquad$ (3.46)

Evenness and self-duality of ${\mathcal{C}}$ ensure that
$\mathcal{Z}_{{\mathcal{C}}}(\tau)$ is invariant under $T$ and $S$
respectively. Indeed, since the code is even, for any
$(\ell_{1},\dots,\ell_{n})\in{\mathcal{C}}$ we have
$\sum_{i}\ell_{i}^{T}g_{\Lambda}^{-1}\ell_{i}\in 2\mathbb{Z}$, and therefore
$T$ is a symmetry of $P_{\mathcal{C}}(\\{\psi_{\ell}\\})$, while it is
invariant under (3.46) because of self-duality (3.38,3.39).

###  3.2 Example: theta series for square glue lattice

For the lattice (2.14) functions $\psi_{\sf g}$ defined in (3.41) read

$\psi_{ab}={1\over|\eta|^{2}}\sum_{k_{1},k_{2}\in\mathbb{Z}}q^{{p\over
4}({a+b\over p}+k_{1}+k_{2})^{2}}\bar{q}^{{p\over 4}({a-b\over
p}+k_{1}-k_{2})^{2}},$ (3.47)

where $(a,b)\in{\sf G}=\mathbb{Z}_{p}\times\mathbb{Z}_{p}$. This can be
written as follows

$|\eta|^{2}\psi_{ab}=\Theta_{a+b,p}\bar{\Theta}_{a-b,p}+\Theta_{a+b-p,p}\bar{\Theta}_{a-b-p,p},$
(3.48)

where

$\Theta_{m,p}=\sum_{n\in\mathbb{Z}}q^{p(n+{m\over 2p})^{2}}.$ (3.49)

These functions are the chiral algebra characters of free boson compactified
at radius $R=\sqrt{2k}$.

Note, if we perform $O(1,1)$ rotation on the lattice $\Lambda$, functions
$\psi_{ab}$ will change. Let

$\Lambda^{\prime}=O\sqrt{p}g,~{}O=\begin{pmatrix}\lambda&0\\\
0&\lambda^{-1}\end{pmatrix}.$ (3.50)

Then

$|\eta|^{2}\psi_{ab}=\sum_{k_{1},k_{2}\in\mathbb{Z}}q^{{p\over 4}({\lambda
a+\lambda^{-1}b\over p}+\lambda k_{1}+\lambda^{-1}k_{2})^{2}}\bar{q}^{{p\over
4}({\lambda a-\lambda^{-1}b\over p}+\lambda k_{1}-\lambda^{-1}k_{2})^{2}}.$
(3.51)

For $\lambda=\sqrt{q}$ with $q\in\mathbb{N}$, we can again decompose
$\psi_{ab}$ as follows

$|\eta|^{2}\psi_{ab}=\sum_{k=0}^{q-1}\Theta_{qa+b+kp,qp}\bar{\Theta}_{qa-b-
kp,qp}+\Theta_{q(a-p)+b+kp,qp}\bar{\Theta}_{q(c_{1}-p)-c_{2}-kp,qp},$ (3.52)

where the functions $\Theta_{m,pq}$ above are now characters of compactified
boson at radius $R=\sqrt{2pq}$.

Finally, if $\lambda=\sqrt{q/r}$ with $q,r$ are co-prime, we can again perform
the same decomposition to obtain a more general result

$\begin{split}|\eta|^{2}\psi_{ab}=\sum_{v_{1}=0}^{r-1}\sum_{v_{2}=0}^{q-1}&\Theta_{q(a+pv_{1})+r(b+pv_{2}),pqr}\bar{\Theta}_{q(a+pv_{1})-r(b+pv_{2}),pqr}+\\\
&\Theta_{q(a+pv_{1})+r(b+pv_{2})-pqr,pqr}\bar{\Theta}_{q(a+pv_{1})-r(b+pv_{2})-pqr,pqr},\end{split}$
(3.53)

where the functions $\Theta_{m,pqr}$ above are characters of compactified
boson at radius $R=\sqrt{2pqr}$.

###  3.3 Partition function in case of isodual codes

In case of isodual codes satisfying (2.26) with pairwise permutation $S$, the
codeword ${\sf c}=({\sf g}_{1},\dots,{\sf g}_{c})\in{\mathcal{C}}$ should be
understood as consisting of $r$ pairs $({\sf g}_{i},{\sf g}_{j})$ with $S$
mapping $i\leftrightarrow j$, while the remaining $c-2r$ “letters” remain
intact. It is convenient to introduce new notation for ${\sf c}$ which is
related to the previous one by permutation,

$\displaystyle{\sf c}=(({\sf g}_{i_{1}},{\sf g}_{j_{1}}),\dots,({\sf
g}_{i_{r}},{\sf g}_{j_{r}}),{\sf g}_{2r+1},\dots,{\sf
g}_{c})\in{\mathcal{C}}.$ (3.54)

With this notation we define an extended enumerator polynomial, which will
depend on both ${\mathcal{C}}$ and $S$. It is a function of $|{\sf G}|^{2}$
variables $y_{{\sf g}_{1}{\sf g}_{2}}$ and $|{\sf G}|$ variables $x_{\sf g}$,

$\displaystyle P_{\mathcal{C}}^{S}(\\{y_{{\sf g}_{1}{\sf g}_{2}}\\},\\{x_{\sf
g}\\})=\sum_{(({\sf g}_{i_{1}},{\sf g}_{j_{1}}),\dots,({\sf g}_{i_{r}},{\sf
g}_{j_{r}}),{\sf g}_{2r+1},\dots,{\sf
g}_{c})\in{\mathcal{C}}}\,\prod_{k=1}^{r}y_{{\sf g}_{i_{k}}{\sf
g}_{j_{k}}}\prod_{i=2r+1}^{c}x_{{\sf g}_{i}}.$ (3.55)

The CFT partition function is given by

$\mathcal{Z}_{{\mathcal{C}}}(\tau)={P^{S}_{\mathcal{C}}(\\{\psi_{{\sf
g}_{1}{\sf g}_{2}}(\tau)\\},\\{\psi_{\sf g}(\tau)\\})},$ (3.56)

where

$\displaystyle\psi_{\ell_{1}\ell_{2}}(\tau)={1\over|\eta(\tau)|^{4}}\sum_{k_{1},k_{2}\in\mathbb{Z}^{2}}\exp(i\pi(v_{1},v_{2})^{T}\tilde{\Omega}(v_{1},v_{2})),$
(3.57) $\displaystyle\tilde{\Omega}=\begin{pmatrix}i\tau_{2}I_{2}&\tau_{1}g\\\
\tau_{1}g&i\tau_{2}I_{2}\end{pmatrix},\qquad
v_{i}=\Lambda^{\perp}\ell_{i}+\Lambda\,k_{i}.$ (3.58)

Under modular transformations $T:\tau\to\tau+1$ and $S:\tau\to-1/\tau$, this
function changes as follows

$\displaystyle T\circ\tilde{\Omega}$ $\displaystyle=$
$\displaystyle\tilde{\Omega}+g\otimes g,$ (3.59) $\displaystyle
S\circ\tilde{\Omega}$ $\displaystyle=$ $\displaystyle-\tilde{\Omega}^{-1}.$
(3.60)

and

$\displaystyle T\circ\psi_{\ell_{1}\ell_{2}}$ $\displaystyle=$
$\displaystyle\exp(2i\pi v_{1}^{T}gv_{2})\psi_{\ell_{1}\ell_{2}},\quad
v_{i}=\Lambda^{\perp}\ell_{i},$ (3.61) $\displaystyle
S\circ\psi_{\ell_{1}\ell_{2}}$ $\displaystyle=$ $\displaystyle{1\over|{\sf
G}|}\sum_{\ell_{1}^{\prime},\ell_{2}^{\prime}\in{\sf
G}=\mathbb{Z}^{2}/g_{\Lambda}}\exp(2i\pi(u_{1}^{T}gv_{1}+u_{2}^{T}gv_{2}))\psi_{\ell_{1}^{\prime}\ell_{2}^{\prime}},\quad
u_{i}=\Lambda^{\perp}\ell_{i}^{\prime}.\ \ \ $ (3.62)

Clearly when ${\mathcal{C}}$ is even in the sense of (2.27) and isodual in the
sense of (2.26), the identities (3.61,3.62) respectively ensure modular
invariance of (3.56).

##  4 Examples: optimal Narain theories for small $c$

In this section we consider a number of explicit examples of code theories. In
particular we discuss optimal theories, i.e. those with the largest spectral
gap, for $c\leq 8$ identified in [7], and show they all are codes theories, in
the sense defined in this paper.

###  4.1 $c=1$

We first consider the simplest case of $n=m=0$, when

$\displaystyle g_{\Lambda}=\left(\begin{array}[]{cc}0&k\\\
k&0\end{array}\right).$ (4.65)

In this case the group ${\sf G}=\mathbb{Z}_{k}\times\mathbb{Z}_{k}$ is
parametrized by vectors $\ell=(a,b)$ for $0\leq a,b<k$. Let’s consider a self-
dual code ${\mathcal{C}}={\mathcal{C}}^{\perp}$ and demand it to be even and
self-dual. When $k$ is prime the only such two codes consist or vectors
$(a,0)$ and $(0,a)$ for $0\leq a<k$ correspondingly. Using appropriate
$O(1,1)$ transformation we can bring corresponding Narain lattice to the form

$\displaystyle\Lambda_{\mathcal{C}}\ni\left(\begin{array}[]{c}a/\sqrt{k}\\\
b\sqrt{k}\end{array}\right),\qquad a,b\in\mathbb{Z}.$ (4.68)

At this point we recognize Narain lattice of a compact boson of radius
$R^{2}=2k$. Choosing $k=1$ will yield boson at self-dual radius, which has
largest possible spectral gap

$\displaystyle\Delta^{*}={1\over 2}.$ (4.69)

The corresponding enumerator polynomial is simply $P=x_{00}$, giving rise to
torus partition function via (3.40) and (3.41),

$\displaystyle\mathcal{Z}_{\mathcal{C}}(\tau,\bar{\tau})=\Psi_{0,0}={1\over|\eta|^{2}}\sum_{n,m\in
Z}q^{(m+n)^{2}\over 4}\bar{q}^{(m-n)^{2}\over
4}={\left|\theta_{3}(2\tau)\right|^{2}+\left|\theta_{2}(2\tau)\right|^{2}\over|\eta|^{2}}.$
(4.70)

Clearly, an appropriate $O(1,1)$ transformation will turn
$\Lambda_{\mathcal{C}}$ to any other Narain lattice in $\mathbb{R}^{1,1}$, or,
equivalently, change the compact boson radius $R$ to any desired value. In
other words, together with the $O(1,1)$ factor our code construction is
versatile enough such that any $c=1$ Narain theory is a code theory. This
emphasizes the bottom-up nature of our approach. While codes are expected to
reflect some algebraic properties of the underlying CFTs in the top-down
constructions [13], in our construction certain non-rational CFTs without
obvious algebraic properties which would make them “finite” also can be
obtained from codes.

###  4.2 $c=2$

We start with $m=2,n=-1,k=2$, which satisfies (2.5) and the glue lattice
generated by

$\displaystyle g_{\Lambda}=\Lambda^{T}g\Lambda=\left(\begin{array}[]{cc}4&\ \
2\\\ 2&-2\\\
\end{array}\right),\quad\Lambda=R\,\,2\left(\begin{array}[]{cc}1&1/2\\\
0&\sqrt{3}/2\end{array}\right),\quad R=\left(\begin{array}[]{cc}1&-1\\\
1&\,\,1\end{array}\right)/\sqrt{2}.$ (4.77)

From the Euclidean point of view this is a hexagonal (triangular) lattice with
the lattice vectors of length $2$, rotated by $\pi/2$. Using equivalence
transformation

$\displaystyle P=\left(\begin{array}[]{cc}0&1\\\ -1&1\\\ \end{array}\right)\in
SL(2,\mathbb{Z})$ (4.80)

we can bring $g_{\Lambda}=\Lambda^{T}g\Lambda$ to the diagonal form

$\displaystyle\left(\begin{array}[]{cc}-2&0\\\ 0&6\\\
\end{array}\right)=P^{T}g_{\Lambda}P,$ (4.83)

which makes decomposition ${\sf G}=\mathbb{Z}_{2}\times\mathbb{Z}_{6}$
manifest, with the map

$\displaystyle{\sf g}=(a,b)\in{\sf G},\quad 0\leq a<2,0\leq
b<6,\qquad\ell({\sf g})=(P^{T})^{-1}\left(\begin{array}[]{c}a\\\
b\end{array}\right)\in\mathbb{Z}^{2}/g_{\Lambda}.$ (4.86)

We consider a code ${\mathcal{C}}$ generated by the following three codewords

$\displaystyle{\sf c_{1}}=((0,3),(1,0)),$ (4.87) $\displaystyle{\sf
c_{2}}=((1,0),(0,1)),$ (4.88) $\displaystyle{\sf c_{3}}=((0,0),(0,2)),$ (4.89)

in the notations ${\sf c}=({\sf g}_{1},{\sf
g}_{2})=((a_{1},b_{1}),(a_{2},b_{2}))$. This codes is iso-dual,
${\mathcal{C}}^{\perp}=S({\mathcal{C}})$, where $S$ is the permutation of two
elements. Corresponding lattice $\Lambda_{\mathcal{C}}$ obtained via (2.13)

$\displaystyle\Lambda_{\mathcal{C}}=\left(\begin{array}[]{cc}\Lambda^{\perp}(P^{T})^{-1}&0\\\
0&\Lambda^{\perp}(P^{T})^{-1}\end{array}\right)\left(\ell+\left(\begin{array}[]{c}2k_{1}\\\
6k_{2}\\\ 2k_{3}\\\
6k_{4}\end{array}\right)\right),\quad\ell^{T}=\sum_{i=1}^{3}n_{i}{\sf
c}_{i},\quad n_{i}\in\mathbb{Z},\quad k_{i}\in\mathbb{Z}.$ (4.96)

In this expression above we should understand codewords ${\sf c}_{i}$ as
regular vectors in $\mathbb{Z}^{4}$. This lattice is a Narain lattice with
respect to the Lorentzian metric (2.28)

$\displaystyle g_{L}=\left(\begin{array}[]{cc}0&S\\\
S&0\end{array}\right),\qquad S=\left(\begin{array}[]{cc}0&1\\\
1&0\end{array}\right).$ (4.101)

By an orthogonal transformation $g_{L}$ can be brought to conventional form

$\displaystyle Og_{L}O^{T}=\left(\begin{array}[]{cc}0&I_{2}\\\
I_{2}&0\end{array}\right),\qquad
O={1\over\sqrt{2}}\left(\begin{array}[]{cccc}1&1&0&0\\\ 0&0&1&-1\\\ 0&0&1&1\\\
-1&1&0&0\\\ \end{array}\right),$ (4.108)

such that $\Lambda_{\mathcal{C}}$ becomes equivalent to the Narain lattice
$\Lambda_{c=2}$ defining $SU(3)_{1}$ WZW theory

$\displaystyle\Lambda_{c=2}$ $\displaystyle\sim$ $\displaystyle
O\Lambda_{C},\quad\Lambda_{c=2}=\left(\begin{array}[]{cc}(\gamma^{-1})^{T}&B\gamma\\\
0&\gamma\end{array}\right),\quad\gamma=\sqrt{b_{2}\over
t_{2}}\begin{pmatrix}1&t_{1}\\\ 0&t_{2}\end{pmatrix},\qquad B={b_{1}\over
b_{2}}\begin{pmatrix}0&1\\\ -1&0\end{pmatrix},$ (4.111)

where $t_{1}+it_{2}=b_{1}+ib_{2}=(1+i\sqrt{3})/2$.

The code enumerator polynomial of ${\mathcal{C}}$ is

$\displaystyle
P^{S}_{\mathcal{C}}=y_{00,00}+y_{00,02}+y_{00,04}+y_{10,01}+y_{10,03}+y_{10,05}+y_{03,10}+y_{03,12}+y_{03,14}+y_{13,11}+y_{13,13}+y_{13,15},$

which yields partition function via (3.56). Shortest lattice vector with
$\ell^{T}={\sf c}_{2}$ or $\ell^{T}={\sf c}_{3}$ and $k_{i}=0$ has length
$|v|^{2}=4/3$, hence corresponding CFT has spectral gap $\Delta^{*}=2/3$.

###  4.3 $c=3,4,5$

Optimal theories for $c=3,4,5$ were constructed from codes in [14]. They
correspond to $k=2$ and $n,m=0$ with

$\displaystyle\Lambda=\sqrt{2}\left(\begin{array}[]{cc}0&1\\\
1&0\end{array}\right),\qquad g_{\Lambda}=2\left(\begin{array}[]{cc}0&1\\\
1&0\end{array}\right),\qquad\Lambda^{\perp}=I_{2}/\sqrt{2}.$ (4.116)

In this case ${\sf G}=\mathbb{Z}_{2}\times\mathbb{Z}_{2}$ which as an additive
group is equivalent to $F_{4}$. As discussed in section 2.3 these codes are
parametrized by B-matrix controlling the B-field of the Narain
compactification, $B^{T}=B\mod 2$, see (2.24). The case of $k=2$ is special
because antisymmetric $B^{T}=-B\mod 2$ and symmetric matrices $B$ are
equivalent.

For the optimal theories with $c=3,4,5$ the symmetric B-matrices, which can be
interpreted as graph adjacency matrix , describes the maximally connected
graph

$\displaystyle B_{ij}=\left\\{\begin{array}[]{c}1,\quad i\neq j,\\\ 0,\quad
i=j.\end{array}\right.$ (4.119)

Their enumerator polynomials and partition functions can be found in [14].
Here we only point out that for $p=2$ and $0\leq a,b<p$

$\displaystyle\psi_{a,b}$ $\displaystyle=$
$\displaystyle{1\over|\eta|^{2}}\sum_{n,m\in\mathbb{Z}}q^{(\tilde{a}+\tilde{b})^{2}/8}\bar{q}^{(\tilde{a}-\tilde{b})^{2}/8},\qquad\tilde{a}=a+2n,\quad\tilde{b}=b+2m,$
(4.120) $\displaystyle\psi_{0,0}$ $\displaystyle=$
$\displaystyle{|\theta_{3}(\tau)|^{2}+|\theta_{4}(\tau)|^{2}\over
2|\eta|^{2}},$ (4.121) $\displaystyle\psi_{1,1}$ $\displaystyle=$
$\displaystyle{|\theta_{3}(\tau)|^{2}-|\theta_{4}(\tau)|^{2}\over
2|\eta|^{2}},$ (4.122) $\displaystyle\psi_{0,1}$ $\displaystyle=$
$\displaystyle\psi_{1,0}={|\theta_{2}(\tau)|^{2}\over 2|\eta|^{2}}.$ (4.123)

in full agreement with [14]. The spectral gaps are $\Delta^{*}=3/4,1,1$ for
$c=3,4,5$ correspondingly.

###  4.4 $c=6,7$

Optimal theories for $c=6,7$ were found in [15] to be related to codes, where
a construction, different from [14, 9], relating codes over $F_{4}$ to CFTs
was introduced. Here we show this construction is a particular case of a more
general construction introduced in this work.

Let us consider the glue matrix

$\displaystyle\Lambda={1\over 3^{1/4}}\left(\begin{array}[]{cc}1&-1\\\
-\sqrt{3}&-\sqrt{3}\end{array}\right).$ (4.126)

This corresponds to $m=-1,n=1,k=0$ case as follows from

$\displaystyle g_{\Lambda}=\left(\begin{array}[]{cc}-2&0\\\
0&2\end{array}\right).$ (4.129)

Clearly ${\sf G}=\mathbb{Z}_{2}\times\mathbb{Z}_{2}$ which can be parametrized
by $\ell^{T}=(a,b)$, $0\leq a,b<2$. As in the previous section we can identify
${\sf G}$ with $F_{4}$ via the Gray map

$\displaystyle(a,b)\rightarrow c(a,b):=a\,\omega+b\,\bar{\omega},$ (4.130)

where $F_{4}=\\{0,\omega,\bar{\omega},1\\}$.

The scalar product inherited from on ${\sf G}$ from (2.3) reads

$\displaystyle\eta((a_{1},b_{1}),(a_{2},b_{2}))=\ell_{1}^{T}g_{\Lambda}^{-1}\ell_{2}={b_{1}b_{2}-a_{1}a_{2}\over
2}.$ (4.131)

Since the scalar product is defined up to integer shits, orthogonality with
respect to $\eta$ is equivalent to orthogonality with respect to

$\displaystyle
a_{1}a_{2}+b_{1}b_{2}\mod\,2=c_{1}c_{2}+\bar{c}_{1}\bar{c}_{2},$ (4.132)

where the right-hand-side uses notations (4.130). This is different from the
conventional Hermitian scalar product on $F_{4}$

$\displaystyle(c_{1},c_{2})=c_{1}\bar{c}_{2}+\bar{c}_{1}c_{2},$ (4.133)

by an additional conjugation. Thus a code ${\mathcal{C}}\in{\sf G}^{c}$ iso-
dual with respect to scalar product on ${\sf G}$ inherited from (2.3) and
pairwise permutation $S$, ${\mathcal{C}}^{\perp}=S({\mathcal{C}})$, will be
isodual to its conjugate, ${\mathcal{C}}^{\perp}=S(\bar{\mathcal{C}})$, with
respect to Hermitian scalar product (4.133). This is exactly the isoduality
condition outlined in [15].

Similarly, the evenness condition (2.27), written in coordinates

$\displaystyle\sum_{i=1}^{c}{b_{i}b_{S(i)}-a_{i}a_{S(i)}\over 2}\mod 2=0,$
(4.134)

matches precisely with the evenness condition of [15].

To complete the comparison with [15] we note that under Construction A (2.13)
group elements $\ell^{T}=(a,b)$ will be mapped to

$\displaystyle v=\Lambda\ell,\quad\Lambda=\left(\begin{array}[]{cc}-{1\over
2}&-{1\over 2}\\\ {\sqrt{3}\over 2}&{-\sqrt{3}\over 2}\end{array}\right),$
(4.137)

which is exactly the map from $c=a\omega+b\bar{\omega}\in F_{4}$ to
$\mathbb{R}^{2}$ used in [15]. In other words, we have shown that the
construction of [15] is exactly the construction of this paper with the glue
matrix taken to be (4.126).

We notice the choice $m=-1,n=1,k=0$ is not the canonical one. By an
appropriate $GL(2,\mathbb{Z})$ transformation we can bring it to
$m=1,n=0,k=2$, satisfying (2.6). The new form of the glue lattice generating
matrix is then

$\displaystyle\Lambda={2\over
3^{1/4}}\left(\begin{array}[]{cc}\frac{1}{2}&1\\\ \frac{\sqrt{3}}{2}&0\\\
\end{array}\right),$ (4.140)

which is a hexagonal (triangular) lattice with the basic vector length
$2/3^{1/4}$.

The codes leading to optimal $c=6$ and $c=7$ theories, the hexacode and the
“septacode” are rather bulky and we do not repeat them here. Let us just
mention that in both cases the resulting spectral gap is
$\Delta^{*}=\sqrt{4/3}$.

###  4.5 $c=8$

We consider the $n=m=0,k=4$ case with the glue lattice

$\displaystyle\Lambda=2\left(\begin{array}[]{cc}0&2^{1/4}\\\ 2^{-1/4}&0\\\
\end{array}\right),\quad g_{\Lambda}=\left(\begin{array}[]{cc}0&4\\\
4&0\end{array}\right),\quad\Lambda^{\perp}=\left(\begin{array}[]{cc}2^{1/4}&0\\\
0&2^{-1/4}\\\ \end{array}\right)/2.$ (4.147)

In this case ${\sf G}=\mathbb{Z}_{4}\times\mathbb{Z}_{4}$ parametrized by
${\sf g}=(a,b)$, $0\leq a,b<4$ and $\ell^{T}=(a,b)$. Let us consider the code
${\mathcal{C}}\in{\sf G}^{8}$ generated by rows of the following matrix

$\displaystyle
G_{\mathcal{C}}=\left(\begin{array}[]{cccccccccccccccc}0&0&0&2&2&0&0&0&0&0&0&0&0&0&0&0\\\
0&0&0&0&2&2&0&0&0&0&0&0&0&0&0&0\\\ 0&0&0&0&0&2&2&0&0&0&0&0&0&0&0&0\\\
1&1&1&1&1&1&1&1&0&0&0&0&0&0&0&0\\\ 2&0&0&0&2&0&0&0&0&0&0&0&0&0&0&0\\\
3&1&0&0&3&1&0&0&2&2&0&0&0&0&0&0\\\ 0&3&1&0&0&3&1&0&0&2&2&0&0&0&0&0\\\
0&0&3&1&0&0&3&1&0&0&2&2&0&0&0&0\\\ 3&0&0&3&1&0&0&3&0&0&0&2&2&0&0&0\\\
1&3&0&0&3&1&0&0&0&0&0&0&2&2&0&0\\\ 0&1&3&0&0&3&1&0&0&0&0&0&0&2&2&0\\\
0&0&0&0&1&1&1&1&1&1&1&1&1&1&1&1\\\ \end{array}\right)$ (4.160)

in the notations ${\sf
c}=(a_{1},\dots,a_{8}|b_{1},\dots,b_{8})\in{\mathcal{C}}$. This code is even
and self-dual with respect to

$\displaystyle\eta=\left(\begin{array}[]{cc}0&I_{8}\\\
I_{8}&0\end{array}\right)/4.$ (4.163)

Accordingly the lattice

$\displaystyle\Lambda_{\mathcal{C}}={1\over
2}\left(\begin{array}[]{cc}2^{1/4}I_{8}&0\\\
0&2^{-1/4}I_{8}\end{array}\right)(G_{\mathcal{C}}^{T}\,z+4k),\quad
z\in\mathbb{Z}^{12},\quad k\in\mathbb{Z}^{16},$ (4.166)

is a Narain lattice with respect to

$\displaystyle g_{L}=\left(\begin{array}[]{cc}0&I_{8}\\\
I_{8}&0\end{array}\right).$ (4.169)

The lattice shortest vector has length $|v|^{2}=2\sqrt{2}$ yielding
$\Delta^{*}=\sqrt{2}$. This follows from the lattice theta series, which can
be readily obtained from the code enumerator polynomial. The code in question
has $2^{16}$ codewords and enumerator polynomial $P_{\mathcal{C}}(x_{ab})$ is
too large to be written explicitly here. Upon substituting
$x_{ab}\rightarrow\psi_{ab}$ where

$\displaystyle\psi_{ab}=\sum_{k\in\mathbb{Z}^{2}}q^{|v|^{2}/2},\qquad
v=\Lambda^{\perp}(\ell+4k),\quad\ell^{T}=(a,b).$ (4.170)

(this definition is different from (3.41) in two ways: i) there is no
$|\eta(\tau)|$ in the denominator because we are interested in the lattice
theta-function rather than the CFT partition function ii) $\psi_{ab}$ depends
on $q$ but not $\bar{q}$ as we are interested in the Euclidean structure
only), we obtain

$\displaystyle
P_{\mathcal{C}}(\psi_{ab})=1+4320t^{2}+61440t^{3}+522720t^{4}+2211840t^{5}+O\left(t^{6}\right),\quad
t=q^{2^{-1/2}},$

which is exactly the theta-function of the Barnes-Wall lattice. This is in
agreement with [21] who identified optimal $c=8$ theory to be based on a
rescaled Barnes-Wall lattice, equipped with the Lorentzian metric and
understood as a Narain lattice.

##  5 Asymptotically large $c$

When $c\gg 1$ asymptotic behavior of spectral gap is not known. Spinless
modular bootstrap bounds $\Delta^{*}/c$ to be less than or equal to
$1/\pi^{2}$ (with this value being obtained numerically) [6], while the full
set of bootstrap constraints is likely to significantly decrease this value.
Averaging over the whole moduli space of Narain theories provides a lower
bound on $\Delta^{*}/c$ to be $1/(2\pi e)$ [21]. Ref. [3] conjectured this
value to be asymptotically saturated,

$\displaystyle\lim_{c\rightarrow\infty}{\Delta^{*}\over c}={1\over 2\pi e}.$
(5.171)

For this to be true, i.e. for the mean value to (asymptotically) be the
largest possible value, the distribution of spectral gaps on the Narain moduli
space for large $c$ must be very sharply peaked around the mean without
outliers. Thus, for consistency, as a necessary condition, variance should be
very small. Using the ensemble of code CFTs, as well as chiral cousins of
Narain theories, ref. [3] has shown the variance of density of states
distribution to be exponentially suppressed $\sim e^{-{\mathcal{O}}(c)}$, the
conclusion consequently confirmed for the Narain theories in [22]. This does
not constitute a proof of (5.171) as variance is not sensitive to possible
outliers.

The conjecture of [3] is based on similarity between the ensemble of codes,
ensemble of sphere packings, and the ensemble (space) of CFTs, and the
problems of maximizing code Hamming distance, density of sphere packing and
CFT spectral gap. Specifically for codes, there is an expectation that the
Gilbert-Varshamov bound (the value resulting from averaging over all codes)
would asymptotically yield the best value of Hamming distance to code size
ratio [23, 24]. Similar expectation holds for the maximal density of lattice
sphere packing: the densest packing to asymptotically saturate the Minkowski
bound, which is simply averaged value over all possible lattices. (For sphere
packings of general kind stronger asymptotic value is expected [25].) While we
leave validity of (5.171) for future studies, here we show there are codes
theories achieving this value of $\Delta^{*}$ for large $c$.

The Construction A used in this paper has a fundamental limitation: the
corresponding lattices have vectors of certain length no matter how big the
dimension $c$ is. This is formalized in equation (2.36), which provides an
upper bound on $\Delta^{*}$. Thus, to obtain $\Delta^{*}$ scaling linearly
with $c$ one has to adjust $n,m,k$ together with $c$ such that $|{\sf G}|$
grows as or faster than $c$. Here for simplicity we focus on the square gluing
lattice $n=m=0$, with prime $k=p$, discussed in sections 2.3, 3.2. The
spectral gap is given by (2.33) with $|v_{\Lambda}|=\sqrt{p}$,

$\displaystyle\Delta^{*}={1\over 2}\min\left({d({\mathcal{C}})},p\right).$
(5.172)

The best (maximal) generalized Hamming distance $d$ beyond small $c$ values is
not known. One nevertheless can bound $d$ from below by consider ensemble
averaging, the so-called Gilbert-Varshamov bound. Then, similarly to the case
of binary linear codes one may expect best $d/c$ to asymptotically approach
the bound when $c\rightarrow\infty$.

By averaged polynomial $\bar{P}(\\{x_{ab}\\})$ we mean enumerator polynomial
averaged over all $p^{c(c-1)/2}$ possible codes parametrized by $B$, see
(2.18). From the CFT point of view this is the calculation of averaged torus
partition function. So far we are interested only in $d$, or alternatively
only in mass but not spin of the lightest non-trivial state, we can consider
torus parameter to be purely imaginary $\tau=i\tau_{2}$. Then function (3.2)
factorizes

$\displaystyle\psi_{ab}(i\tau_{2})={1\over|\eta(\tau)|^{2}}\psi_{a}\psi_{b},\qquad\psi_{a}=\Theta_{2a,p}(i\tau_{2}/2)=\sum_{k\in\mathbb{Z}}e^{-\pi\tau_{2}(a+kp)^{2}/p}.$
(5.173)

Going back to enumerator polynomial, instead of variables $x_{ab}$ we use

$\displaystyle x_{ab}=t_{a}t_{b},\qquad t_{a}=t_{-a},$ (5.174)

where the last property reflects $\psi_{a}=\psi_{-a}$. We conjecture the form
of corresponding averaged enumerator polynomial based on invariance under
MacWilliams identity and explicit checks for sufficiently small $n$ and prime
$p$, for which direct evaluation of $\bar{P}(\\{t_{a}t_{b}\\})$ using computer
algebra is feasible,

$\displaystyle\bar{P}(\\{t_{a}t_{b}\\})$ $\displaystyle=$
$\displaystyle{1\over
p^{c(c-1)/2}}\sum_{B}P_{{\mathcal{C}}(B)}(\\{t_{a}t_{b}\\})=$ (5.175)
$\displaystyle
t_{0}^{2c}+{\sum\limits_{k=0}^{p=1}\left(\sum\limits_{a=0}^{p-1}\sum\limits_{b=0}^{p-1}\cos\left({2\pi
kab\over
p}\right)t_{a}t_{b}\right)^{c}-p\,t_{0}^{c}\left(\sum\limits_{a=0}^{p-1}t_{a}\right)^{c}\over
p^{c}}.$

Now we are ready to analyze this expression to deduce the lower bound on
$\Delta^{*}$. For large $c$, the main contribution to (5.175) comes from
$k=0$, leading to the averaged partition function

$\displaystyle\bar{Z}\approx{1\over|\eta|^{2c}}{\left(\sum\limits_{a=0}^{p-1}\psi_{a}\right)^{2c}\over
p^{c}}={1\over|\eta|^{2c}}{\left(\sum\limits_{n\in\mathbb{Z}}e^{-\pi\tau_{2}n^{2}/p}\right)^{2c}\over
p^{c}}.$ (5.176)

Interpreted as sum over lattice points, the numerator is simply the sum over
$2c$-dimensional square lattice of size $1/\sqrt{p}$. On the length scales of
$\sim 1/\sqrt{p}$ or larger this is just the homogeneous distribution of
points with the averaged density $1/(1/\sqrt{p}^{2c})=p^{c}$. This factor
exactly cancels $p^{c}$ in the denominator of (5.176) and we find density of
states

$\displaystyle\rho(\Delta)={(2\pi)^{c}\Delta^{c-1}\over\Gamma(c)}$ (5.177)

valid on scales $\Delta\gtrsim 1/\sqrt{p}$. This is exactly the density of
states of “$U(1)$-gravity” – Narain theory averaged over the whole moduli
space. Accordingly, the threshold for the density to become of order one is
$\Delta=c/(2\pi e)$, which is our Gilbert-Varshamov bound. For this result to
be valid we must require $p/2>c/(2\pi e)$, otherwise shortest vector of
$\Lambda_{\mathcal{C}}$ would have length $\sqrt{p}$.

To conclude, for sufficiently large $p$ we find that the averaged density of
states (with zero chemical potential for spin) of $n=m=0$, and prime $k=p$
code theories is the same as the averaged density of states for all Narain
theories. In particular in the limit $c\rightarrow\infty$, for $p>c/(\pi e)$
there are code theories with $\Delta^{*}/c=1/(2\pi e)$. Provided the
conjecture of [3] is correct, it would mean similar conjecture applies to
$n=m=0$, prime $k=p$ codes, in the sense that their averaged Hamming distance
is asymptotically the best one.

##  6 Discussion

In this paper we proposed a family of constructions mapping additive codes
over abelian groups ${\sf G}=\mathbb{Z}_{p}\times\mathbb{Z}_{q}$ to Narain
lattices and hence Narain CFTs. Each construction is parametrized by a triple
of integer numbers $n,m,k$ and an element from $O(1,1)$ parameterizing an even
“glue” lattice $\Lambda\subset\mathbb{R}^{1,1}$. The resulting Narain lattice
$\Lambda_{\mathcal{C}}$ associated with a code ${\mathcal{C}}\subset{\sf
G}^{c}$ obeys

$\displaystyle\underbrace{\Lambda\oplus\dots\oplus\Lambda}_{c\,{\rm
times}}\subset\Lambda_{\mathcal{C}}\subset\underbrace{\Lambda^{\perp}\oplus\dots\oplus\Lambda^{\perp}}_{c\,{\rm
times}}\subset\mathbb{R}^{c,c}.$ (6.178)

We call this glue construction following [18]. This construction generalizes
and encompasses those of [14, 9], [15] and [16]. We call the CFTs obtained
from codes “code theories.” Their torus partition functions $Z_{\mathcal{C}}$
are given in terms of the so-called code enumerator polynomials, which are
multi-variable polynomials satisfying certain algebraic identities, which
guarantee modular invariance of $Z_{\mathcal{C}}$. In this way one can
construct many new solutions to modular bootstrap constraints.

We have shown that all conjectural optimal Narain theories for $c\leq 8$
identified in [7], meaning those with the largest spectral gap of
$U(1)^{c}\times U(1)^{c}$ primaries are code theories. Furthermore we have
shown there are code theories with the spectral gap $\Delta^{*}$ scaling
linearly with $c\gg 1$, $\Delta^{*}\propto c/(2\pi e)$, with the coefficient
which has been conjectured in [3] to be maximal possible. The message of our
work is clear: we conjecture that optimal Narain theories for any $c$ are code
theories, either following from the constructions outlined in this paper, or
their possible generalizations.

Speaking of the latter, one can straightforwardly generalize our construction
by starting with a glue lattice $\Lambda\subset\mathbb{R}^{3,3}$ or in fact
$\Lambda\subset\mathbb{R}^{r,r}$ for any $r\geq 2$. Another important
direction would be to connect the bottom-up approach of this paper with the
top-down approach of [13] where quantum codes were given an interpretation in
terms of CFT Hilbert space extended by defect operators. Finally, given our
conjecture that optimal theories are code theories, it would be interesting to
develop our approach into a practical way of constructing optimal theories
with $c>8$, thus complementing conventional modular bootstrap. This would be
an important but challenging task because there is no known efficient methods
to construct “good” codes with largest or even large (generalized) Hamming
distance. And though there is a finite number of codes for any given ${\sf G}$
and $c$, their number grows exponentially with $c$. Furthermore, there is an
infinite number of constructions, i.e. infinite number of different ${\sf G}$
and $\Lambda$, making the problem seemingly incomprehensible. This pessimistic
assessment could be too naive, we expect only finite number of constructions
to be relevant for any given $c$. The inequality (2.36) as well as the results
of section 5 clearly indicate $|{\sf G}|$ can not be too small, $|{\sf
G}|^{1/2}\geq ac$, for $c\gg 1$ with some positive constant $a$. We also
strongly suspect large generalized Hamming distance, for given $c$, would
require $|{\sf G}|$ not be too large. We conjecture this may come from the
linear programming constraints stemming from the algebraic identities
satisfied by code enumerator polynomial (the MacWilliam identity and the
condition due to code evenness), i.e. generalization of Delsarte’s bounds [26]
to the types of codes of interest. For $c\gg 1$ we expect the bound to have
the form $|{\sf G}|^{1/2}\leq bc$ with some $b>a$. Thus for large but finite
$c$ we expect large but finite number of glue groups satisfying $bc\geq|{\sf
G}|^{1/2}\geq ac$. This form of the bound on $|{\sf G}|$ is merely a guess;
the important point here is the expectation that the problem of identifying
the code with largest generalized Hamming distance can be reduced to an
optimization problem over a finite set. Of course even for moderate $c$ naive
brute-force approaches such as going through all possible codes very quickly
becomes unfeasible. The resulting optimization problem over a discrete set
would be NP-hard, but novel quantum platforms promise an exciting hope of
solving medium-sized discrete optimization problems in real time, the avenue
we hope to pursue in the future [27].

To conclude, we would like to point out another very important direction for
future work – to extend the connection between codes and CFTs beyond the
Narain theories.

Acknowledgments. The authors are supported by the National Science Foundation
under Grant No. PHY 2013812. DC was also partially supported by the National
Science Foundation under the Grant No. PHY 1818878. AD thanks Institut des
Hautes $\acute{\rm E}$tudes Scientifiques, where this work was completed, for
hospitality.

##  Appendix A Shortest vector bound

Let us consider a two-dimensional Euclidean lattice $\sf\Lambda$ with the
scalar product $g_{2}$. Using rotation and up to an overall rescaling basis
vectors can be chosen to be $1$ and $\tau=\tau_{1}+i\tau_{2}$, where we
introduced complex coordinates on $\mathbb{R}^{2}$. In other words

$\displaystyle
g_{2}={\sf\Lambda}^{T}{\sf\Lambda},\qquad{\sf\Lambda}=\alpha\left(\begin{array}[]{cc}1&\tau_{1}\\\
0&\tau_{2}\end{array}\right),$ (1.181)

and $\alpha$ is some scalar factor. Using $GL(2,\mathbb{Z})$ transformations,
together with an appropriate rotation and rescaling, we can bring $\tau$ to
belong to fundamental domain

$\displaystyle|\tau_{1}|\leq 1/2,\quad\tau_{2}\geq 0,\quad|\tau|\geq 1.$
(1.182)

In this case the shortest vector is $\alpha(1,0)^{T}$ and its norm is
$\alpha^{2}$. From (1.181) we find $\alpha^{4}={\rm det}g_{2}/\tau_{2}^{2}$
and from (1.182) we know $\tau_{2}^{2}\geq 3/4$. We therefore find the bound

$\displaystyle\alpha^{2}\leq{2\over\sqrt{3}}\sqrt{{\rm det}g_{2}},$ (1.183)

which in many cases is conservative. Applying that to (2.35) we obtain (2.36).

## References

  * [1] S. Hellerman, _A universal inequality for CFT and quantum gravity_ , _Journal of High Energy Physics_ 2011 (2011) .
  * [2] T. Hartman, C. A. Keller and B. Stoica, _Universal spectrum of 2d conformal field theory in the large c limit_ , _Journal of High Energy Physics_ 2014 (2014) 1.
  * [3] A. Dymarsky and A. Shapere, _Comments on the holographic description of narain theories_ , _Journal of High Energy Physics_ 2021 (2021) .
  * [4] T. Hartman, D. Mazáč and L. Rastelli, _Sphere packing and quantum gravity_ , _Journal of High Energy Physics_ 2019 (2019) 48.
  * [5] H. Cohn and N. Elkies, _New upper bounds on sphere packings i_ , _Annals of Mathematics_ (2003) 689.
  * [6] N. Afkhami-Jeddi, H. Cohn, T. Hartman, D. de Laat and A. Tajdini, _High-dimensional sphere packing and the modular bootstrap_ , _arXiv preprint arXiv:2006.02560_ (2020) .
  * [7] N. Afkhami-Jeddi, H. Cohn, T. Hartman and A. Tajdini, _Free partition functions and an averaged holographic duality_ , _Journal of High Energy Physics_ 2021 (2021) 1.
  * [8] G. Nebe, E. M. Rains and N. J. A. Sloane, _Self-dual codes and invariant theory_ , vol. 17. Springer, 2006.
  * [9] A. Dymarsky and A. Shapere, _Solutions of modular bootstrap constraints from quantum codes_ , _Physical Review Letters_ 126 (2021) 161602\.
  * [10] L. Dolan, P. Goddard and P. Montague, _Conformal field theories, representations and lattice constructions_ , _Communications in mathematical physics_ 179 (1996) 61.
  * [11] J. Henriksson, A. Kakkar and B. McPeak, _Classical codes and chiral cfts at higher genus_ , 2021.  10.48550/ARXIV.2112.05168.
  * [12] J. Henriksson, A. Kakkar and B. McPeak, _Narain cfts and quantum codes at higher genus_ , 2022.  10.48550/ARXIV.2205.00025.
  * [13] M. Buican, A. Dymarsky and R. Radhakrishnan, _Quantum codes, cfts, and defects_ , _arXiv preprint arXiv:2112.12162_ (2021) .
  * [14] A. Dymarsky and A. Shapere, _Quantum stabilizer codes, lattices, and cfts_ , _Journal of High Energy Physics_ 2021 (2021) 1.
  * [15] A. Dymarsky and A. Sharon, _Non-rational narain cfts from codes over f4_ , _Journal of High Energy Physics_ 2021 (2021) 1.
  * [16] S. Yahagi, _Narain cfts and error-correcting codes on finite fields_ , _arXiv preprint arXiv:2203.10848_ (2022) .
  * [17] Y. Furuta, _Relation between spectra of narain cfts and properties of associated boolean functions_ , _arXiv preprint arXiv:2203.11643_ (2022) .
  * [18] J. H. Conway and N. J. A. Sloane, _Sphere packings, lattices and groups_ , vol. 290. Springer Science & Business Media, 2013.
  * [19] J. Henriksson, A. Kakkar and B. McPeak, _Codes, lattices, and cfts at higher genus_ , _arXiv preprint arXiv:2112.05168_ (2021) .
  * [20] J. Henriksson, A. Kakkar and B. McPeak, _Narain cfts and quantum codes at higher genus_ , _arXiv preprint arXiv:2205.00025_ (2022) .
  * [21] N. Afkhami-Jeddi, H. Cohn, T. Hartman and A. Tajdini, _Free partition functions and an averaged holographic duality_ , _arXiv preprint arXiv:2006.04839_ (2020) .
  * [22] S. Collier and A. Maloney, _Wormholes and spectral statistics in the narain ensemble_ , _arXiv preprint arXiv:2106.12760_ (2021) .
  * [23] M. V. Kozlov, _The correcting capacities of linear codes_ , in _Doklady Akademii Nauk_ , vol. 186, pp. 275–278, Russian Academy of Sciences, 1969.
  * [24] J. Pierce, _Limit distribution of the minimum distance of random linear codes_ , _IEEE Transactions on Information Theory_ 13 (1967) 595\.
  * [25] S. Torquato and F. H. Stillinger, _New conjectural lower bounds on the optimal density of sphere packings_ , _Experimental Mathematics_ 15 (2006) 307.
  * [26] P. Delsarte, _An algebraic approach to the association schemes of coding theory_ , _Philips Res. Rep. Suppl._ 10 (1973) vi+.
  * [27] A. Dymarsky, A. Kakkar and I. Refat, _work in progress_ , .