# Unsupervised Translation Quality Estimation
Exploiting Synthetic Data and Pre-trained Multilingual Encoder

Yuto Kuroda† Atsushi Fujita‡ Tomoyuki Kajiwara† Takashi Ninomiya†
†Ehime University
‡National Institute of Information and Communications Technology
<EMAIL_ADDRESS>kajiwara<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Translation quality estimation (TQE) is the task of predicting translation
quality without reference translations. Due to the enormous cost of creating
training data for TQE, only a few translation directions can benefit from
supervised training. To address this issue, unsupervised TQE methods have been
studied. In this paper, we extensively investigate the usefulness of synthetic
TQE data and pre-trained multilingual encoders in unsupervised sentence-level
TQE, both of which have been proven effective in the supervised training
scenarios. Our experiment on WMT20 and WMT21 datasets revealed that this
approach can outperform other unsupervised TQE methods on high- and low-
resource translation directions in predicting post-editing effort and human
evaluation score, and some zero-resource translation directions in predicting
post-editing effort.

##  1 Introduction

Translation quality estimation (TQE) (Blatz et al., 2004; Specia et al., 2018)
is the task to predict translation quality of an output of machine translation
(MT) system with respect to the source sentence. Developing TQE methods that
correlate well with human evaluation can support human decision-making:
whether to use the MT output as is, to post-edit it, or to use other MT
systems. Hence, TQE is practically important to advance the real-world use of
MT systems.

Methods for TQE have been actively studied mainly in the WMT TQE shared task
Specia et al. (2020, 2021); Zerva et al. (2022). Most of them are supervised
methods, i.e., the TQE models are trained on tuples of source sentence, MT
output, and manually labeled quality score. However, creating a TQE dataset,
i.e., a set of such tuples, is costly because the quality score can only be
obtained through manual work, such as human evaluation for DA (direct
assessment) score or post-editing for HTER (Human-targeted Translation Edit
Rate) (Snover et al., 2006), carried out by annotators who have high
competence in both source and target languages. Furthermore, the distribution
of translation quality and errors can vary widely from one MT system to
another; thus it is impractical to tailor such a dataset for each MT system.
As such, even in the latest WMT22 TQE task (Zerva et al., 2022), human-labeled
training data are available only for eleven translation directions.

To address this issue, unsupervised TQE methods have been studied. As reviewed
in Section 2, past work has exploited pre-trained multilingual encoders and
neural MT (NMT) systems, as well as bilingual parallel data. Synthetic TQE
data generated from monolingual or parallel data have also been exploited in
order to adapt given pre-trained encoders to the TQE task (Tuan et al., 2021;
Eo et al., 2021). However, no study has yet made an extensive investigation
into the impact of the synthetic TQE data on relatively recent pre-trained
multilingual encoders that are prevalent in the supervised training scenarios
(Ranasinghe et al., 2020; Wang et al., 2021; Rei et al., 2022).

This paper investigates the usefulness of synthetic TQE data in unsupervised
training of sentence-level TQE models on top of relatively recent pre-trained
multilingual encoders. Our experiment on WMT20 and WMT21 datasets shows that
this approach can outperform other unsupervised TQE methods on high- and low-
resource language pairs. Even though each component method in this study is
not novel, it makes the first comparison of representative options for
unsupervised TQE.

##  2 Previous Work

In this paper, we regard TQE methods that do not use any human-labeled pairs
of source sentence and an MT output for it as “unsupervised.”

Existing unsupervised TQE methods rely on either multilingual encoders or NMT
models. Unsupervised methods based on a multilingual encoders compute cosine
similarity between sentence embeddings of source sentence and MT output as the
TQE score Fonseca et al. (2019); Song et al. (2021). For example, TQE using
only LaBSE (Feng et al., 2022) can be considered as an unsupervised method,
since LaBSE has been trained through fine-tuning mBERT (Devlin et al., 2019)
on parallel data, where mBERT is pre-trained exclusively on monolingual data
in multiple languages. Sentence embeddings obtained by LaBSE are dominated by
language-specific information, and thus hinder precise estimation of semantic
similarity across different languages (Tiyajamorn et al., 2021). DREAM
(Tiyajamorn et al., 2021) and MEAT (Kuroda et al., 2022) address this issue by
disentangling sentence embeddings to meaning and language embeddings, and
outperformed LaBSE in the sentence-level TQE task by comparing meaning
embeddings.

In addition to parallel data, synthetic TQE data have also been exploited for
TQE. Tuan et al. (2021) generated synthetic TQE data from a parallel corpus
using either an MT system or a masked language model. Eo et al. (2021) did so
from monolingual data in the target language via round-trip MT. However,
existing work has not assessed the impact of synthetic TQE data on training
models with multilingual encoders; Tuan et al. (2021) evaluated only the
predictor-estimator model based on bidirectional LSTM (Kim et al., 2017). Eo
et al. (2021) trained TransQuest with a relatively new multilingual encoder,
i.e., XLM-RLarge (Conneau et al., 2020), but did not evaluate their method on
human-labeled data. Given the promising results in the supervised TQE
scenarios obtained with more sophisticated pre-trained multilingual encoders
(Ranasinghe et al., 2020; Specia et al., 2020, 2021; Wang et al., 2021; Zerva
et al., 2022; Rei et al., 2022), it is worth investigating whether such models
can also be trained well in a fully unsupervised fashion, i.e., without using
any human-labeled TQE data.

Other unsupervised TQE methods rely on NMT models trained on parallel data.
While Fomicheva et al. (2020) used internal weights of NMT models, Prism
(Thompson and Post, 2020) used only the forced-decoding score, i.e., the
likelihood of MT output given source sentence computed by the given NMT model,
as the TQE score. Publicly available pre-trained multilingual NMT models, such
as mBART50 (Tang et al., 2021) and M2M-100 (Fan et al., 2022), extend the
applicability of this approach to many translation directions.

##  3 Training with Synthetic TQE Data

In this paper, we primarily focus on HTER Snover et al. (2006) as the
sentence-level quality metric. HTER is the edit rate of MT outputs with
respect to their human post-edited versions. HTER has drawn less attention
(Tuan et al., 2021; Eo et al., 2021); most of past work targeted DA scores
(Fomicheva et al., 2020; Thompson and Post, 2020; Tiyajamorn et al., 2021;
Kuroda et al., 2022). Nevertheless, HTER is also a useful indicator in the
translation production workflow where MT output is regarded as a draft
translation (ISO/TC37, 2017) and thus has been tackled in WMT Specia et al.
(2020, 2021); Zerva et al. (2022). We also assume that we can obtain synthetic
TQE data that approximate it as explained below. In contrast, there is no such
a way for DA scores.

#### Generation of synthetic TQE data:

Following Tuan et al. (2021), we generate synthetic TQE data from a bilingual
parallel corpus, $C=(x_{i},y_{i})_{i=1}^{N}$. First, using an MT system, we
translate the source sentences of the parallel corpus, $x_{i}$, into the
target language. We then compute translation edit rate (TER) Snover et al.
(2006) for each MT output, $y^{\prime}_{i}$, and its corresponding human
translation in the parallel corpus, $y_{i}$. Finally, we compose synthetic TQE
data, comprising
$(x_{i},y^{\prime}_{i},\mathrm{TER}(y_{i},y^{\prime}_{i}))_{i=1}^{n}$. Here,
we assume that TER would approximate HTER even though $y$ and $y^{\prime}$ are
separately generated.

#### TQE model:

We investigate neural TQE models consisting of a pre-trained multilingual
encoder and additional layers. In particular, we compare the following two
encoding methods, following Ranasinghe et al. (2020).

Split:

Source sentence and MT output are separately encoded with a prepended [CLS]
token. In other words, we encode “[CLS] source sentence” and “[CLS] MT output”
respectively, concatenate the two embeddings each corresponding to the [CLS]
token, and feed the following layers with it.

Concat:

Source sentence and MT output are jointly encoded with a prepended [CLS] token
and a [SEP] token in between. That is, we encode only “[CLS] source sentence
[SEP] MT output” and input the embedding corresponding to the [CLS] token to
the following layers.

In both methods, we input the embedding(s) corresponding to the [CLS] token(s)
to the following multi-layer perceptron (MLP), and train a regression model
through minimizing root mean squared error (RMSE).

##  4 Experiment

We evaluated the usefulness of synthetic TQE data with the methods presented
in Section 3 on the WMT20 and WMT21 TQE datasets (Specia et al., 2020, 2021;
Fomicheva et al., 2022). As the metrics to predict, we used both HTER and DA
score.

###  4.1 Setting

#### Test sets for HTER:

The WMT20 TQE Task 2 dataset111https://statmt.org/wmt20/quality-estimation-
task.html includes two high-resource translation directions: English-to-German
(En$\rightarrow$De) and English-to-Chinese (En$\rightarrow$Zh). Instead of
official test sets for which gold HTER scores were not disclosed, we used the
official development sets consisting of $1$k tuples for our evaluation. For
the WMT21 TQE Task 2 dataset,222https://statmt.org/wmt21/quality-estimation-
task.html we used all the test sets for 11 translation directions, each
consisting of $1$k tuples of source sentence, an MT output for it, and HTER
score obtained through post-editing the MT output. Note that lower HTER
indicates higher quality of MT output.

#### Test sets for DA score:

We used the WMT20 TQE Task 1 and WMT21 TQE Task 1 datasets. The former
contains $1$k tuples of source sentece, an MT output for it, and a DA score
for each of six translation directions, while the latter consists of another
sets of $1$k tuples for the same six translation directions and five
translation directions for which no parallel data are available for TQE. The
DA score has been computed by standardizing 0–100 scores given by multiple
annotators. Note that higher DA score indicates higher MT quality.

#### Classification of translation directions:

The MT outputs for each translation direction had been generated by a
Transformer model (Vaswani et al., 2017) trained using the fairseq
toolkit333https://github.com/pytorch/fairseq (Ott et al.,
2019).444https://github.com/facebookresearch/mlqe/tree/main/nmt_models
Henceforth, according to the size of bilingual parallel corpora used to train
the MT models (Table 1) and we exploit for synthesizing TQE data, we denote
En$\rightarrow$De and En$\rightarrow$Zh as high-resource, Romanian-to-English
(Ro$\rightarrow$En) and Estonian-to-English (Et$\rightarrow$En) as medium-
resource, Nepali-to-English (Ne$\rightarrow$En) and Sinhalese-to-English
(Si$\rightarrow$En) as low-resource, and the rest as zero-shot555Note that
bilingual parallel data for these language pairs could have been used for pre-
training. translation directions.

Language pair | Bilingual | Synthetic
---|---|---
| En | – | De | 23,360,441 | 22,268,661
| En | – | Zh | 20,305,268 | 19,811,665
| Ro | – | En | 3,901,501 | 3,178,631
| Et | – | En | 877,769 | 863,973
| Ne | – | En | 498,271 | 477,563
| Si | – | En | 646,766 | 585,598
Table 1: The number of sentence pairs in the bilingual parallel corpora and
synthetic TQE data.

#### Synthetic TQE data:

To generate synthetic TQE data, we used
M2M-100666https://huggingface.co/facebook/m2m100_418M (Fan et al., 2022) and
the bilingual parallel corpora officially provided by the task organizers
(Table 1) on which the aforementioned MT systems had been trained. Before
decoding the source-side of the parallel corpora, we fine-tuned M2M-100 for
each translation direction. First, we sampled $1$M, $200$k, and $50$k sentence
pairs for high-, medium-, and low-resource language pairs, respectively, from
the bilingual parallel corpora (Table 1), where we limited parallel sentences
to those whose each side was composed of up to $128$ sub-word tokens. We then
fine-tuned M2M-100 on the sampled parallel sentences with the AdamW optimizer
(Loshchilov and Hutter, 2019) ($\beta_{1}=0.9$, $\beta_{2}=0.999$,
$\epsilon=1\times 10^{-8}$) on batches of $16$ sentence pairs with a learning
rate of $3\times 10^{-5}$, and terminated training after one epoch for each of
high- and medium-resource language pairs and three epochs for each of low-
resource language pairs. When decoding the source-side of the bilingual
parallel corpora, we performed beam search with beam size of 5, length penalty
of 1.0, and maximum sequence length of 200. After decoding, we calculated TER
of the MT output with respect to the target-side of the parallel corpus using
SacreBLEU777https://github.com/mjpost/sacrebleu (Post, 2018), and composed
synthetic tuples of source sentence, MT output, and the TER score, discarding
those with TER score greater than $1.0$. See Table 1 for the results.

Model | $P$ | $A$
---|---|---
LaBSE | 470M | -
MEAT | 470M | 1M
M2M-100 | 490M | -
XLM-RBase+MLP (split) | 280M | 1,537
XLM-RLarge+MLP (split) | 560M | 2,049
InfoXLMBase+MLP (split) | 280M | 1,537
LaBSE+MLP (split) | 470M | 1,537
XLM-RBase+MLP (concat) | 280M | 769
XLM-RLarge+MLP (concat) | 560M | 1,025
InfoXLMBase+MLP (concat) | 280M | 769
LaBSE+MLP (concat) | 470M | 769
Table 2: Number of parameters: $P$ for the pre-trained model and $A$ for the
additional MLP component(s).

Model | HTER (WMT20) | HTER (WMT21)
---|---|---
En$\rightarrow$De | En$\rightarrow$Zh | En$\rightarrow$De | En$\rightarrow$Zh | Ro$\rightarrow$En | Et$\rightarrow$En | Ne$\rightarrow$En | Si$\rightarrow$En
LaBSE | 0.104 | 0.238 | 0.119 | 0.122 | 0.748 | 0.547 | 0.451 | 0.420
MEAT | 0.216 | 0.384 | 0.208 | 0.215 | 0.763 | 0.614 | 0.516 | 0.501
M2M-100 (w/o FT) | 0.323 | 0.333 | 0.269 | 0.189 | 0.808 | 0.679 | 0.248 | 0.386
M2M-100 (w/ FT) | 0.361 | 0.340 | 0.287 | 0.200 | 0.718 | 0.674 | 0.394 | 0.389
XLM-RBase+MLP (split) | 0.393 | 0.451∗ | 0.280 | 0.149 | 0.347 | 0.390 | 0.391 | 0.466
XLM-RLarge+MLP (split) | 0.389 | 0.457∗ | 0.271 | 0.182 | 0.390 | 0.446 | 0.448 | 0.503
InfoXLMBase+MLP (split) | 0.420∗ | 0.452∗ | 0.321 | 0.178 | 0.364 | 0.411 | 0.446 | 0.478
LaBSE+MLP (split) | 0.423∗ | 0.466∗ | 0.324 | 0.194 | 0.407 | 0.432 | 0.466 | 0.527
XLM-RBase+MLP (concat) | 0.456∗ | 0.492∗ | 0.282 | 0.180 | 0.593 | 0.480 | 0.434 | 0.536
XLM-RLarge+MLP (concat) | 0.455∗ | 0.542∗ | 0.297 | 0.214 | 0.643 | 0.583 | 0.601∗ | 0.608∗
InfoXLMBase+MLP (concat) | 0.506∗ | 0.524∗ | 0.384∗ | 0.212 | 0.572 | 0.530 | 0.534 | 0.591∗
LaBSE+MLP (concat) | 0.483∗ | 0.548∗ | 0.352∗ | 0.218 | 0.649 | 0.569 | 0.598∗ | 0.611∗

Table 3: Pearson’s $r$ with HTER for non-zero-shot translation directions:
bold and “$\ast$” respectively indicate the highest value in each column and a
significant improvement over all the baselines.

Model | HTER (WMT21)
---|---
Ru$\rightarrow$En | En$\rightarrow$Cs | En$\rightarrow$Ja | Km$\rightarrow$En | Ps$\rightarrow$En
LaBSE | 0.317 | 0.065 | 0.144 | 0.395 | 0.339
MEAT | 0.402 | 0.196 | 0.198 | 0.535 | 0.424
M2M-100 (w/o FT) | 0.339 | 0.265 | 0.154 | 0.406 | 0.336
XLM-RBase+MLP (split) | 0.228 | 0.290 | 0.151 | 0.386 | 0.233
XLM-RLarge+MLP (split) | 0.315 | 0.327∗ | 0.173 | 0.407 | 0.237
InfoXLMBase+MLP (split) | 0.291 | 0.330∗ | 0.126 | 0.401 | 0.176
LaBSE+MLP (split) | 0.245 | 0.315 | 0.138 | 0.425 | 0.219
XLM-RBase+MLP (concat) | 0.379 | 0.298 | 0.150 | 0.507 | 0.322
XLM-RLarge+MLP (concat) | 0.398 | 0.301 | 0.145 | 0.580∗ | 0.457
InfoXLMBase+MLP (concat) | 0.362 | 0.379∗ | 0.207 | 0.566 | 0.428
LaBSE+MLP (concat) | 0.357 | 0.370∗ | 0.170 | 0.584∗ | 0.407

Table 4: Pearson’s $r$ with HTER for zero-shot translation directions: bold
and “$\ast$” follow those in Table 4.

#### TQE models:

As the pre-trained multilingual encoder, we compared XLM-
RBase888https://huggingface.co/xlm-roberta-base (Conneau et al., 2020), XLM-
RLarge999https://huggingface.co/xlm-roberta-large,
InfoXLMBase101010https://huggingface.co/microsoft/infoxlm-base (Chi et al.,
2021), and LaBSE111111https://huggingface.co/sentence-transformers/LaBSE (Feng
et al., 2022). The last two have optimized the embedding for [CLS] tokens
during pre-training. Table 2 shows the numbers of parameters of the examined
TQE model. We trained each TQE model with the AdamW optimizer (Loshchilov and
Hutter, 2019) ($\beta_{1}=0.9$, $\beta_{2}=0.999$, $\epsilon=1\times 10^{-8}$)
on batches of 128 tuples of synthetic TQE data for all the six translation
directions, also updating the parameters of the pre-trained multilingual
encoder. We performed early stopping for training with a patience of $3$,
according to a validation loss calculated for every $10$k steps on a randomly
sub-sampled $10\%$ tuples of training data. We separately trained four models
with different learning rates of $5\times 10^{-5}$, $1\times 10^{-5}$,
$5\times 10^{-6}$, and $1\times 10^{-6}$, and selected the best model for
evaluation according to the validation loss.

Model | Direct Assessment (WMT20)
---|---
En$\rightarrow$De | En$\rightarrow$Zh | Ro$\rightarrow$En | Et$\rightarrow$En | Ne$\rightarrow$En | Si$\rightarrow$En
LaBSE | 0.084 | 0.036 | 0.705 | 0.550 | 0.547 | 0.455
MEAT | 0.213 | 0.223 | 0.718 | 0.585 | 0.633 | 0.570
M2M-100 (w/o FT) | 0.246 | 0.209 | 0.807 | 0.669 | 0.298 | 0.398
M2M-100 (w/ FT) | 0.254 | 0.279 | 0.780 | 0.680 | 0.497 | 0.418
XLM-RBase+MLP (split) | 0.230 | 0.218 | 0.329 | 0.367 | 0.386 | 0.408
XLM-RLarge+MLP (split) | 0.208 | 0.231 | 0.407 | 0.436 | 0.392 | 0.427
InfoXLMBase+MLP (split) | 0.242 | 0.238 | 0.362 | 0.437 | 0.410 | 0.419
LaBSE+MLP (split) | 0.219 | 0.217 | 0.386 | 0.455 | 0.420 | 0.421
XLM-RBase+MLP (concat) | 0.231 | 0.236 | 0.535 | 0.433 | 0.430 | 0.462
XLM-RLarge+MLP (concat) | 0.183 | 0.256 | 0.590 | 0.526 | 0.541 | 0.479
InfoXLMBase+MLP (concat) | 0.286 | 0.285 | 0.502 | 0.486 | 0.524 | 0.491
LaBSE+MLP (concat) | 0.252 | 0.264 | 0.569 | 0.527 | 0.605 | 0.510

Table 5: Pearson’s $r$ with DA score for non-zero-shot translation directions:
bold and “$\ast$” follow those in Table 4.

Model | Direct Assessment (WMT21)
---|---
En$\rightarrow$De | En$\rightarrow$Zh | Ro$\rightarrow$En | Et$\rightarrow$En | Ne$\rightarrow$En | Si$\rightarrow$En
LaBSE | 0.089 | 0.099 | 0.718 | 0.591 | 0.585 | 0.360
MEAT | 0.176 | 0.270 | 0.722 | 0.607 | 0.670 | 0.459
M2M-100 (w/o FT) | 0.212 | 0.250 | 0.805 | 0.652 | 0.357 | 0.384
M2M-100 (w/ FT) | 0.211 | 0.321 | 0.767 | 0.651 | 0.534 | 0.367
XLM-RBase+MLP (split) | 0.187 | 0.320 | 0.373 | 0.348 | 0.489 | 0.375
XLM-RLarge+MLP (split) | 0.172 | 0.360 | 0.429 | 0.416 | 0.508 | 0.392
InfoXLMBase+MLP (split) | 0.226 | 0.333 | 0.388 | 0.364 | 0.521 | 0.392
LaBSE+MLP (split) | 0.211 | 0.340 | 0.444 | 0.404 | 0.547 | 0.386
XLM-RBase+MLP (concat) | 0.195 | 0.364 | 0.578 | 0.429 | 0.478 | 0.418
XLM-RLarge+MLP (concat) | 0.153 | 0.370∗ | 0.628 | 0.510 | 0.626 | 0.431
InfoXLMBase+MLP (concat) | 0.265∗ | 0.385∗ | 0.539 | 0.447 | 0.586 | 0.450
LaBSE+MLP (concat) | 0.259 | 0.382∗ | 0.625 | 0.514 | 0.669 | 0.465

Table 6: Pearson’s $r$ with DA score for non-zero-shot translation directions:
bold and “$\ast$” follow those in Table 4.

Model | Direct Assessment (WMT21)
---|---
Ru$\rightarrow$En | En$\rightarrow$Cs | En$\rightarrow$Ja | Km$\rightarrow$En | Ps$\rightarrow$En
LaBSE | 0.387 | 0.131 | 0.126 | 0.357 | 0.363
MEAT | 0.524 | 0.269 | 0.181 | 0.531 | 0.458
M2M-100 (w/o FT) | 0.410 | 0.350 | 0.121 | 0.349 | 0.388
XLM-RBase+MLP (split) | 0.335 | 0.230 | 0.135 | 0.371 | 0.232
XLM-RLarge+MLP (split) | 0.400 | 0.253 | 0.189 | 0.402 | 0.248
InfoXLMBase+MLP (split) | 0.361 | 0.237 | 0.173 | 0.387 | 0.194
LaBSE+MLP (split) | 0.335 | 0.248 | 0.139 | 0.414 | 0.207
XLM-RBase+MLP (concat) | 0.433 | 0.238 | 0.130 | 0.487 | 0.338
XLM-RLarge+MLP (concat) | 0.466 | 0.240 | 0.155 | 0.508 | 0.455
InfoXLMBase+MLP (concat) | 0.392 | 0.312 | 0.166 | 0.504 | 0.434
LaBSE+MLP (concat) | 0.425 | 0.324 | 0.179 | 0.520 | 0.412

Table 7: Pearson’s $r$ with DA score for zero-shot translation directions: bold and “$\ast$” follow those in Table 4. Model | #TD | HTER (WMT20) | HTER (WMT21)
---|---|---|---
En$\rightarrow$De | En$\rightarrow$Zh | En$\rightarrow$De | En$\rightarrow$Zh | Ro$\rightarrow$En | Et$\rightarrow$En | Ne$\rightarrow$En | Si$\rightarrow$En
InfoXLMBase+MLP (concat) | 6 | 0.506∗ | 0.524∗ | 0.384∗ | 0.212∗ | 0.572∗ | 0.530∗ | 0.534∗ | 0.591
InfoXLMBase+MLP (concat) | 1 | 0.407 | 0.435 | 0.284 | 0.155 | 0.491 | 0.296 | 0.341 | 0.593
LaBSE+MLP (concat) | 6 | 0.483 | 0.548∗ | 0.352 | 0.218∗ | 0.649∗ | 0.569 | 0.598∗ | 0.611
LaBSE+MLP (concat) | 1 | 0.466 | 0.475 | 0.344 | 0.184 | 0.511 | 0.602∗ | 0.268 | 0.586
Table 8: Pearson’s $r$ with HTER of LaBSE+MLP and InfoXLMBase+MLP trained on
different numbers of translation directions (#TD): bold indicates the winner
and “$\ast$” shows the value is significantly higher than the other.

#### Baselines:

Our baselines include three existing unsupervised TQE methods (four models)
that do not use synthetic TQE data but bilingual parallel data. LaBSE is the
one explained in Section 2. While MEAT121212https://github.com/kuro961/MEAT
was trained on a smallish parallel corpus in the original work (Kuroda et al.,
2022), we trained it on the entire bilingual corpora in Table 1. As the
forced-decoding methods, we compared the original and fine-tuned M2M-100
models. The latter, denoted as “w/ FT,” were six direction-specific models
that are also used for generating synthetic TQE data.

#### Evaluation metric:

Accuracy of each TQE model was measured with Pearson’s product-moment
correlation coefficient (Pearson’s $r$) between its outputs and gold scores.
The baseline methods are designed to give a higher score for a better MT
output, unlike HTER. Therefore, we multiplied $-1$ to all their predicted
scores before computing Pearson’s $r$ with the gold HTER. We did the same for
the predicted scores of the models trained on synthetic TQE data before
computing Pearson’s $r$ with the DA score. Statistical significance between
coefficients ($p<0.05$) was tested with Williams significance test using mt-
qe-eval.131313https://github.com/ygraham/mt-qe-eval

###  4.2 Results on Predicting HTER

Table 4 shows our experimental results on the non-zero-shot translation
directions. Some neural TQE models with the “concat” setting outperformed the
baselines on the high- and low-resource translation directions. When compared
the two variants based on the same pre-trained models, the “concat” model was
consistently superior to the “split” model, suggesting the usefulness of
capturing cross-lingual token-level correspondences during encoding, as
demonstrated in Ranasinghe et al. (2020). InfoXLMBase consistently
outperformed XLM-RBase. Considering that both have the same number of
parameters and identical vocabulary (see Table 2), this indicates that the
[CLS] embedding optimized during pre-training InfoXLMBase helps TQE. We
consider that LaBSE also benefits from the optimized embedding for [CLS]. For
most test sets, the larger XLM-R (XLM-RLarge+MLP) outperformed its smaller
counterpart (XLM-RBase+MLP), thanks to the larger number of parameters.

TQE models trained on synthetic data show deteriorated results in the medium-
resource translation directions, i.e., Ro$\rightarrow$En and
Et$\rightarrow$En.141414Previous work (Tiyajamorn et al., 2021; Kuroda et al.,
2022) on predicting DA scores also shows a smaller improvement on these two
translation directions compared to others. Similarly, the performance of fine-
tuned M2M-100 was worse than the original M2M-100 on these two translation
directions. To delve the reasons for it, we need to untangle the following
possible factors: (a) size of synthetic training data, (b) language-specific
features, (c) domain mismatch between parallel data and test data, (d)
differences in the quality of the hypothesis, and (e) differences in post-edit
quality.

Table 4 shows the results on the zero-shot translation directions. The
“concat” version of TQE models outperformed baselines on En$\rightarrow$CS and
Km$\rightarrow$En, indicating that synthetic data also help zero-shot TQE
depending on translation direction. Similarly to the results in Table 4, we
observed that the following factors affect performance: (a) encoding methods
(“split” models vs. “concat” models), (b) optimization of [CLS] during pre-
training (XLM-RBase+MLP (concat) vs. InfoXLMBase+MLP (concat)), and (c) the
number of parameters (XLM-RBase+MLP (concat) vs. XLM-RLarge+MLP (concat)).

###  4.3 Results on Predicting DA Scores

Preceding sections have focused on HTER as the target metric to predict,
assuming that labels of synthetic data, i.e., TER, should be correlated with
HTER. To investigate performance in another evaluation metric that is less
correlated with HTER, we also evaluated TQE models on predicting DA scores.

Tables 7 and 7 show experimental results on six translation directions in the
WMT20 and WMT21 TQE Task 1 test sets, respectively. InfoXLMBase+MLP (concat)
predicted DA scores on high-resource translation directions better than all
the baseline models. In contrast, for middle-resource and low-resource
translation directions, none of the neural TQE models trained on synthetic
data outperformed MEAT, except for LaBSE+MLP (concat) for Si$\rightarrow$En in
WMT21 test set.

Table 7 shows the results on the zero-shot translation directions. XLM-
RLarge+MLP (concat) and LaBSE+MLP (concat) outperformed two of three baselines
in all the translation directions except for En$\rightarrow$Cs, but did not
outperform MEAT.

These results demonstrate that exploiting parallel corpora, even for other
language pairs, in the way of what MEAT does, contributed to the prediction of
DA score more than synthetic data labeled with TER scores. We speculate that
this stems from the low reliability of TER score computed from separately
produced MT output and reference translation and the low relevance between TER
and DA score.

##  5 Impact of Multi-directional Training

Previous work has revealed that neural language models and NMT models can
benefit from training data for multiple languages and multiple translation
directions (Conneau et al., 2020; Tang et al., 2021).

Our investigation demonstrates that it also applies to TQE models, as shown in
Table 8. We took LaBSE+MLP (concat) and InfoXLMBase+MLP (concat) according to
their superior results in Table 4, compared two variants of these models
trained exclusively on synthetic TQE data for multiple (#TD=6, Section 4) or
single (#TD=1, (Fomicheva et al., 2020; Tuan et al., 2021; Eo et al., 2021))
translation directions on the same TQE task as in Section 4, and confirmed
that the former outperformed the latter on most translation directions.

##  6 Conclusion

In this paper, we investigated the usefulness of synthetic training data for
TQE, in particular for predicting HTER using pre-trained multilingual
encoders. Our experiment on WMT20 and WMT21 TQE datasets confirmed that the
TQE models trained on synthetic TQE data better predict HTER on high- and low-
resource translation directions, compared to existing unsupervised TQE methods
that do not use synthetic TQE data.

Our work has the following contributions that have never been examined in
previous work: (a) we found pre-trained multilingual encoders with optimized
embedding for [CLS], i.e., LaBSE and InfoXLM, can be trained well exclusively
on synthetic TQE data, (b) we reconfirmed that joint encoding of source
sentence and MT output works better than separate encoding as in the
supervised setting (Ranasinghe et al., 2020), (c) we identified the positive
impact of using multi-directional synthetic TQE data in unsupervised TQE
training, and (d) synthetic data with TER are also useful for predicting DA
score on high- and low-resource translation directions.

## Limitations

Our experiment covers only eleven translation directions, and our results do
not guarantee the same conclusions on other translation directions. As
demonstrated in our experiment, even for the same translation direction (two
En$\rightarrow$De and two En$\rightarrow$Zh tasks in WMT20 and WMT21), the TQE
accuracy can be visibly different. This implies that the difficulty of the
task depends on the characteristics of the test data, the MT systems used for
generating MT outputs, human annotators recruited for obtaining gold HTER, and
so forth.

The entire experiment was carried out on a single V100 GPU. If we had a more
powerful environment, higher TQE accuracy could be achieved by employing
larger pre-trained multilingual encoders, larger batch sizes, longer training,
and so forth.

## Ethics Statement

As shown in our experiment, predicted scores does not perfectly correlate with
the gold score obtained through human post-editing. Therefore, such predicted
scores could mislead potential users. Note that this is not specific to our
work, but common in the TQE task.

## Acknowledgments

This work was supported by JSPS KAKENHI Grant Number JP20K19861. These
research results were obtained from the commissioned research (No.22501) by
National Institute of Information and Communications Technology (NICT), Japan.

## References

  * Blatz et al. (2004) John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004.  Confidence Estimation for Machine Translation.  In _Proceedings of the 20th International Conference on Computational Linguistics_ , pages 315–321.
  * Chi et al. (2021) Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021.  InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3576–3588.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised Cross-lingual Representation Learning at Scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Eo et al. (2021) Sugyeong Eo, Chanjun Park, Hyeonseok Moon, Jaehyung Seo, and Heuiseok Lim. 2021.  Dealing with the Paradox of Quality Estimation.  In _Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021)_ , pages 1–10.
  * Fan et al. (2022) Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. 2022.  Beyond English-Centric Multilingual Machine Translation.  _Journal of Machine Learning Research_ , 22(1):4839–4886.
  * Feng et al. (2022) Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022.  Language-agnostic BERT Sentence Embedding.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics_ , pages 878–891.
  * Fomicheva et al. (2022) Marina Fomicheva, Shuo Sun, Erick Fonseca, Chrysoula Zerva, Frédéric Blain, Vishrav Chaudhary, Francisco Guzmán, Nina Lopatina, Lucia Specia, and André F. T. Martins. 2022.  MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 4963–4974.
  * Fomicheva et al. (2020) Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. 2020.  Unsupervised Quality Estimation for Neural Machine Translation.  _Transactions of the Association for Computational Linguistics_ , 8:539–555.
  * Fonseca et al. (2019) Erick Fonseca, Lisa Yankovskaya, André F. T. Martins, Mark Fishel, and Christian Federmann. 2019.  Findings of the WMT 2019 Shared Tasks on Quality Estimation.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2)_ , pages 1–10.
  * ISO/TC37 (2017) ISO/TC37. 2017.  ISO 18587:2017 translation services: Post-editing of machine translation output: Requirements.
  * Kim et al. (2017) Hyun Kim, Jong-Hyeok Lee, and Seung-Hoon Na. 2017.  Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation.  In _Proceedings of the Second Conference on Machine Translation_ , pages 562–568.
  * Kuroda et al. (2022) Yuto Kuroda, Tomoyuki Kajiwara, Yuki Arase, and Takashi Ninomiya. 2022.  Adversarial Training on Disentangling Meaning and Language Representations for Unsupervised Quality Estimation.  In _Proceedings of the 29th International Conference on Computational Linguistics_ , pages 5240–5245.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled Weight Decay Regularization.  In _Proceedings of the 7th International Conference on Learning Representations_.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A Fast, Extensible Toolkit for Sequence Modeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 48–53.
  * Post (2018) Matt Post. 2018.  A Call for Clarity in Reporting BLEU Scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191.
  * Ranasinghe et al. (2020) Tharindu Ranasinghe, Constantin Orasan, and Ruslan Mitkov. 2020.  TransQuest: Translation Quality Estimation with Cross-lingual Transformers.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 5070–5081.
  * Rei et al. (2022) Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and André F. T. Martins. 2022.  CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task.  In _Proceedings of the Seventh Conference on Machine Translation (WMT)_ , pages 634–645.
  * Snover et al. (2006) Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. 2006.  A Study of Translation Edit Rate with Targeted Human Annotation.  In _Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers_ , pages 223–231.
  * Song et al. (2021) Yurun Song, Junchen Zhao, and Lucia Specia. 2021.  SentSim: Crosslingual Semantic Evaluation of Machine Translation.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3143–3156.
  * Specia et al. (2020) Lucia Specia, Frédéric Blain, Marina Fomicheva, Erick Fonseca, Vishrav Chaudhary, Francisco Guzmán, and André F. T. Martins. 2020.  Findings of the WMT 2020 Shared Task on Quality Estimation.  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 743–764.
  * Specia et al. (2021) Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary, and André F. T. Martins. 2021.  Findings of the WMT 2021 Shared Task on Quality Estimation.  In _Proceedings of the Sixth Conference on Machine Translation_ , pages 684–725.
  * Specia et al. (2018) Lucia Specia, Carolina Scarton, and Gustavo Henrique Paetzold. 2018.  Quality Estimation for Machine Translation.  _Synthesis Lectures on Human Language Technologies_ , 11(1):1–162.
  * Tang et al. (2021) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2021.  Multilingual Translation from Denoising Pre-Training.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3450–3466.
  * Thompson and Post (2020) Brian Thompson and Matt Post. 2020.  Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing_ , pages 90–121.
  * Tiyajamorn et al. (2021) Nattapong Tiyajamorn, Tomoyuki Kajiwara, Yuki Arase, and Makoto Onizuka. 2021.  Language-agnostic Representation from Multilingual Sentence Encoders for Cross-lingual Similarity Estimation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7764–7774.
  * Tuan et al. (2021) Yi-Lin Tuan, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Francisco Guzmán, and Lucia Specia. 2021.  Quality Estimation without Human-labeled Data.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 619–625.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017.  Attention is All You Need.  In _Proceedings of the 31st Conference on Neural Information Processing Systems_ , pages 5998–6008.
  * Wang et al. (2021) Jiayi Wang, Ke Wang, Boxing Chen, Yu Zhao, Weihua Luo, and Yuqi Zhang. 2021.  QEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task.  In _Proceedings of the Sixth Conference on Machine Translation_ , pages 948–954.
  * Zerva et al. (2022) Chrysoula Zerva, Frédéric Blain, Ricardo Rei, Piyawat Lertvittayakumjorn, José G. C. de Souza, Steffen Eger, Diptesh Kanojia, Duarte Alves, Constantin Orăsan, Marina Fomicheva, André F. T. Martins, and Lucia Specia. 2022.  Findings of the WMT 2022 Shared Task on Quality Estimation.  In _Proceedings of the Seventh Conference on Machine Translation (WMT)_ , pages 69–99.

Model | $\mathit{lr}$ | Time [h]
---|---|---
MEAT | $1\times 10^{-6}$ | 24
M2M-100 (w/ FT) |  |
En$\rightarrow$De |  $3\times 10^{-5}$ ($\ast$) | 9
En$\rightarrow$Zh |  $3\times 10^{-5}$ ($\ast$) | 10
Ro$\rightarrow$En |  $3\times 10^{-5}$ ($\ast$) | 2
Et$\rightarrow$En |  $3\times 10^{-5}$ ($\ast$) | 2
Ne$\rightarrow$En |  $3\times 10^{-5}$ ($\ast$) | 2
Si$\rightarrow$En |  $3\times 10^{-5}$ ($\ast$) | 2
XLM-RBase+MLP (split) | $1\times 10^{-5}$ | 556
XLM-RLarge+MLP (split) | $1\times 10^{-5}$ | 962
InfoXLMBase+MLP (split) #TD=6 | $1\times 10^{-5}$ | 273
LaBSE+MLP (split) | $1\times 10^{-5}$ | 443
XLM-RBase+MLP (concat) | $1\times 10^{-5}$ | 549
XLM-RLarge+MLP (concat) | $1\times 10^{-5}$ | 1105
InfoXLMBase+MLP (concat) #TD=6 | $1\times 10^{-5}$ | 386
LaBSE+MLP (concat) #TD=6 | $1\times 10^{-5}$ | 272
InfoXLMBase+MLP (concat) #TD=1 |  |
En$\rightarrow$De | $5\times 10^{-5}$ | 69
En$\rightarrow$Zh | $5\times 10^{-5}$ | 70
Ro$\rightarrow$En | $1\times 10^{-5}$ | 61
Et$\rightarrow$En | $5\times 10^{-5}$ | 20
Ne$\rightarrow$En | $1\times 10^{-5}$ | 49
Si$\rightarrow$En | $1\times 10^{-6}$ | 12
LaBSE+MLP (concat) #TD=1 |  |
En$\rightarrow$De | $1\times 10^{-5}$ | 80
En$\rightarrow$Zh | $5\times 10^{-5}$ | 65
Ro$\rightarrow$En | $1\times 10^{-5}$ | 46
Et$\rightarrow$En | $1\times 10^{-5}$ | 22
Ne$\rightarrow$En | $5\times 10^{-5}$ | 30
Si$\rightarrow$En | $5\times 10^{-6}$ | 15

Table 9:  Best learning rate ($\mathit{lr}$) and the total computation time
including learning rate search: ($\ast$) indicates that this parameter is not
searched.

Translation direction | Time [h]
---|---
En$\rightarrow$De | 963
En$\rightarrow$Zh | 856
Ro$\rightarrow$En | 173
Et$\rightarrow$En | 31
Ne$\rightarrow$En | 6
Si$\rightarrow$En | 7

Table 10: Time spent for decoding parallel data for generating synthetic TQE
data.

##  Appendix A Computation Time and Learning Rate

For the entire experiment, a single V100 GPU was used. See Table 10 for the
learning rate and training time. We also show the time spent for decoding
source-side of the bilingual parallel corpora for generating synthetic TQE
data in Table 10.

The best learning rate was determined among four candidate values. However, we
performed just one training run for each learning rate value. When fine-tuning
M2M-100 for each translation direction, we used a fixed learning rate value
and performed it only once.


# Exciton condensation in biased bilayer graphene

Harley D. Scammell  School of Physics, the University of New South Wales,
Sydney, NSW, 2052, Australia  Oleg P. Sushkov  School of Physics, the
University of New South Wales, Sydney, NSW, 2052, Australia

###### Abstract

We consider suspended bilayer graphene under applied perpendicular electric
bias field that is known to generate a single particle gap $2\Delta$ and a
related electric polarization ${\cal P}$. We argue that the bias also drives a
quantum phase transition from band insulator to superfluid exciton condensate.
The transition occurs when the exciton binding energy exceeds the band gap
$2\Delta$. We predict the critical bias (converted to band gap),
$\Delta_{c}\approx 60$ meV, below which the excitons condense. The critical
temperature, $T_{c}(\Delta)$, is maximum at $\Delta\approx 25$ meV,
$T_{c}^{\text{max}}\approx 115$ K, decreasing significantly at smaller
$\Delta$ due to thermal screening. Entering the condensate phase, the
superfluid transition is accompanied by a cusp in the electric polarization
${\cal P}(\Delta)$ at $\Delta\to\Delta_{c}$, which provides a striking
testable signature. Additionally, we find that the condensate prefers to form
a pair density wave.

Introduction — Excitonic condensates in two-dimensional (2D) materials promise
novel superfluid [1, 2, 3, 4, 5] or topological [6, 7, 8, 9, 10, 11]
properties, and have thereby attracted considerable theoretical and
experimental attention. Moreover, due to these novel transport behaviours,
excitonic condensates promise a route to future technological advancements. So
far, however, experimental realisations of the desired condensate have proven
problematic.

Theoretically it is convenient to classify exciton condensates (also known as
exciton insulators) by the nature of the corresponding interaction-driven
quantum phase transition. Class I corresponds to the (semi)metal-to-exciton
condensate phase transition [12, 13, 14]. In this case there are simultaneous
Fermi surfaces of electrons and holes. If the Fermi surfaces are identical, an
arbitrarily weak attraction between electrons and holes leads to condensation.
This situation is analogous to BCS superconductivity. Class II corresponds to
the band insulator-to-exciton condensate phase transition. In this case there
is no Fermi surface and the interaction must exceed a critical value to
generate the condensate.

Graphene layers, with their very near particle-hole symmetry, have provided a
hunting ground for exciton condensation. Previous theoretical considerations
include bilayer graphene (BLG) — unbiased with AB stacking [15, 16, 17] or
biased with AA stacking [18, 19]. There have been no corresponding
experimental detections of condensation.

Another set of proposals are graphene double layers [20, 21, 22, 23, 24] or
double bilayers [25, 26, 27, 28], separated by a dielectric. Theoretical
predictions for the Berezinskii–Kosterlitz–Thouless (BKT) transition
temperature for such systems vary significantly from room temperature in Ref.
[20] to 1 mK in Refs. [23, 24] — the key difference arises due to the
inclusion [23, 24] or exclusion [20] of Coulomb screening. Experimentally,
there is one recent indirect indication of possible zero-magnetic field
exciton condensation in double-bilayer graphene with WSe2 spacer [29] and also
in an InAs/GaSb bilayer [30], with both scenarios belonging to Class I. On the
other hand, there have been several experimental reports of exciton
condensation in quantum Hall regime in strong magnetic field for double-layer
graphene [31, 32, 33] or other double layer systems [34, 35, 36]; such
excitonic pairing occurs between different Landau levels.

Most, if not all, previous studies of exciton condensation have been aimed at
class I. However, class I is necessarily a many-body problem and progress
often requires uncontrolled approximations. Here, instead, we consider
condensation in class II. A striking technical advantage is that we only need
to consider a two-body problem. In class II the condition of exciton
condensation is the equality of the exciton binding energy $\epsilon_{b}$ to
the single particle band gap, $2\Delta$. The present work is focused on biased
BLG. Unbiased BLG is a semimetal, however, application of an electric bias
(perpendicular electric field) opens a single particle band gap $2\Delta$ [37]
– placing it within class II. The bias is created by symmetric metallic gates
above and below the plane.

A careful treatment of the screened attractive electron-hole Coulomb
interaction is essential to make quantitative predictions of the binding
energy and hence of the condensation transition. To this end, we account for
three sources of screening: (i) Screening by metallic gates placed a distance
$d$ above and below the BLG plane; (ii) dielectric screening due to a material
between BLG and the gates; and (iii) BLG self-screening and retardation
thereof, as captured via the Random Phase Approximation (RPA). We find that
all three sources of screening significantly influence, i.e. reduce, the
condensation critical temperature. To this end, we propose idealised
experimental setups to allow to maximise the critical temperature. A key
development in this work is our treatment of the retardation of the self-
screening, i.e. retardation of the screened Coulomb potential.

The key development of this work is that we are able to: use reliable two-body
techniques to describe excitonic bounds states in a band insulator; account
for environmental sources of screening; treat dynamically
screening/retardation. This combination affords a reliable prediction of the
critical temperature, and how it may be optimised with respect to system
parameters. Entering the exciton condensate phase, we lose this quantitative
control over the problem, and instead resort to mean-field theory. We note
that the exciton condensate is a neutral superfluid, and that experimentally
distinguishing between this and the band insulator phase is subtle. Using our
mean-field description, we establish two key experimental signatures of the
excitonic condensate.

Figure 1: Screening factor vs momentum for $\Delta=10$meV, d=1000nm,
$\epsilon_{r}=1$. The factor is presented for two imaginary frequencies,
$\xi=0$, $\xi=2\Delta$, and for two temperatures, $T=0$, $T=3\Delta/4=87$ K.

Methods — We consider biased bilayer graphene for which the low-energy single
particle Hamiltonian can be reduced to [38]

$\displaystyle H_{0}=\left(\begin{array}[]{c
c}\Delta-\mu&-\frac{p_{-}^{2}}{2m}\\\
-\frac{p_{+}^{2}}{2m}&-\Delta-\mu\end{array}\right).$ (3)

$H_{0}$ is written in terms of $\\{A_{1},B_{2}\\}$ orbitals, with $A,B$
referring to graphene sublattice and subscripts $1,2$ referring to layers.
Here $p_{\pm}=\tau p_{x}\pm ip_{y}$, $\bm{p}$ is the in-plane momentum,
$\tau=\pm 1$ the valley quantum number, $m\approx 0.032m_{e}$ the effective
mass, and $\Delta$ is proportional to the bias electric field, $\Delta\propto
E$, [39]. The chemical potential $\mu$ is set to zero (half-filling) for the
rest of this work. There are corrections to (3) related to electron-hole
asymmetry, trigonal warping, etc. However, influence of all these corrections
on the exciton is negligible at $\Delta<60$meV, see Ref.[40], so here we
disregard the corrections. Hence electron and hole dispersions, denoted
$\omega_{\bm{p}}^{(\pm)}$, are symmetric about $\mu=0$, i.e.
$\omega_{\bm{p}}^{(\pm)}=\pm\omega_{\bm{p}}$, with
$\omega_{\bm{p}}=\sqrt{\Delta^{2}+p^{4}/(4m^{2})}$. Application of the bias
induces electric polarisation along the field. If $a\approx 0.3$nm is the
separation between the planes and $e$ is the electron charge, the electric
dipole moment per unit area is $|e|a{\cal P}_{\Delta}$, with the layer
polarization ${\cal P}_{\Delta}$, due to the bias $\propto\Delta$, given by

$\displaystyle{\cal
P}_{\Delta}=4\int\frac{d^{2}p}{(2\pi)^{2}}(|\beta^{(-)}_{\bm{p}}|^{2}-|\alpha^{(-)}_{\bm{p}}|)=\frac{2m\Delta}{\pi}\ln\frac{\Lambda}{|\Delta|}.$
(4)

The factor 4 is due to the spin and valley degeneracy,
$\alpha^{(-)}_{\bm{p},\tau}$ and $\beta^{(-)}_{\bm{p},\tau}$ are upper and
lower components of the negative energy eigenfunction (for a given valley),
and $\Lambda$ is the ultraviolet energy cutoff. We confirm via a direct
calculation that including the next two orbitals, i.e. $\\{A_{2},B_{1}\\}$,
naturally cuts off of the UV divergence; the choice $\Lambda=0.5$ eV in (4) is
consistent with the direct four-orbital calculation.

Figure 2: Inter- and intra-valley excitons: intravalley have zero total
momentum $\bm{P}=\bm{0}$, while intervalley have $\bm{P}=2\bm{K}$, with
$\bm{K}$ the valley momentum.

Attraction between electron and hole is due to the screened Coulomb
interaction

$\displaystyle V_{\bm{q},i\xi}=-\frac{2\pi
e^{2}}{\epsilon_{r}q/\Upsilon_{q}-2\pi e^{2}\Pi({\bm{q}},i\xi,T)}$ (5)

Here $\bm{q}$ is the momentum transfer, $\xi$ is the imaginary frequency
transfer, and $T$ is temperature. Other parameters are: (i)
$\Upsilon_{p}=\tanh(pd)$ accounts for the metallic gate screening, at a
distance $d$ above and below the BLG plane; (ii) $\epsilon_{r}$ is the
dielectric constant of the substrate/superstrate material between BLG and
gates; and (iii) $\Pi({\bm{q}},i\xi,T)$ is the polarisation operator of BLG,
with details presented in the Supplement [41]. To demonstrate key features of
(5), we introduce a screening factor – defined as the ratio of the screened
interaction (5) to the bare Coulomb interaction, $-2\pi
e^{2}/(\epsilon_{r}q)$. The screening factor provides a measure of the
effectiveness of screening. Fig. 1 demonstrates that the screening very
strongly depends on frequency and on temperature. Notably, thermally excited
electrons practically fully screen the static interaction, i.e. the screening
factor becomes vanishingly small.

Figure 3:  (a) Exciton binding energy at $T=0$ and three values of distance to
gates, $d=20,100,1000$ nm vs the band gap, at $\epsilon_{r}=1$. Dashed lines
correspond to LSE and solid lines to BSE. (b) Superfluid domes, for the
following sets of parameters: {$\epsilon_{r}=1$, $d=1000$nm},
{$\epsilon_{r}=1$, $d=100$nm}, {$\epsilon_{r}=3.9$, $d=1000$nm}. Points
represent $T_{c}$ versus the band gap parameter $\Delta$, as computed from the
BSE (6).

A frequency dependence of $V_{\bm{q},i\xi}$ corresponds to retardation of the
interaction in the time-domain. For a weakly-bound exciton in an insulator,
with binding energy $\epsilon_{b}\ll 2\Delta$, retardation is not important
[40]. However, retardation becomes essential for strongly-bound excitons
$\epsilon_{b}\sim 2\Delta$. We point out that the condition for exciton
condensation, $\epsilon_{b}=2\Delta$, necessarily implies strongly-bound
excitons and therefore retardation is important. To properly treat retardation
in the exciton binding problem, one has to employ the Bethe-Salpeter equation
(BSE) [42]

$\displaystyle\chi_{\xi_{n},\bm{k}}=-T\sum_{m}\int\frac{d^{2}k^{\prime}}{(2\pi)^{2}}\frac{V_{{\bm{k}-\bm{k}^{\prime}},i(\xi_{n}-\xi_{m})}Z_{\bm{k,k^{\prime}}}^{\tau^{\prime},\tau}}{(E/2-\omega_{\bm{k}})^{2}+\xi_{n}^{2}}\chi_{\xi_{m},\bm{k}^{\prime}},$
(6)

written in terms of the amputated two-particle Green’s function
$\chi_{\xi_{n},\bm{k}}$. Here $E=2\Delta-\epsilon_{b}$, $\xi_{n}=(2n+1)\pi T$
and
$Z_{\bm{k,k^{\prime}}}^{\tau^{\prime},\tau}=\langle\psi^{(-)}_{\bm{k},\tau^{\prime}}|\psi^{(-)}_{\bm{k}^{\prime},\tau^{\prime}}\rangle\langle\psi^{(+)}_{\bm{k}^{\prime},\tau}|\psi^{(+)}_{\bm{k},\tau}\rangle$
is the vertex form factor, with $\psi^{(\pm)}_{\bm{k},\tau}$ denoting the
single particle wavefunctions for conduction and valence bands of (3), see
also [41]. The total momentum of the electron and hole is encoded in the
valley indices, and is either zero for $\tau=\tau^{\prime}$ (intravalley
pairing) or non-zero for $\tau=-\tau^{\prime}$ (intervalley pairing) – as
depicted in Fig. 2. The form factors do not distinguish spin, yet they weakly
distinguish between intra- and intervalley pairing; the implications for the
condensate phase is discussed later. The interaction has an SU(2)$\times$SU(2)
spin-symmetry – correspondingly, the exciton bound state has an SO(4) spin
symmetry relating the spin-singlet and triplet configurations.

We pause to mention that under an instantaneous interaction,
$V_{{\bm{k}-\bm{k}^{\prime}}}$, e.g. as obtained by ignoring the frequency
dependence of screening, the exciton can be described by a wave function that
obeys the Lippmann-Schwinger equation (LSE),

$\displaystyle(E_{0}-2\omega_{\bm{k}})\Psi_{\bm{k}}=\int\frac{d^{2}k^{\prime}}{(2\pi)^{2}}V_{{\bm{k}-\bm{k}^{\prime}}}Z_{\bm{k,k^{\prime}}}^{\tau^{\prime},\tau}\tanh\left(\frac{\omega_{\bm{k}^{\prime}}}{2T}\right)\Psi_{\bm{k}^{\prime}}.$
(7)

LSE is a linear eigenvalue problem and can be easily solved to find the
eigenenergy $E_{0}$ and eigenfunction $\Psi_{\bm{k}}$; solution of (7) shows
that the Lippmann-Schwinger wave function $\Psi_{\bm{k}}$ is well localised in
the momentum space, $k\lesssim\sqrt{2m\Delta}$. This corresponds to the
exciton spatial size $r\sim 1/\sqrt{2m\Delta}$. We see a hint that the
wavefunction approach is problematic at $\Delta\to 0$. We denote the binding
energy computed from LSE as $\epsilon_{b}^{0}=2\Delta-E_{0}$.

Phase diagram — Via direct computation of Eq. (6), the intervalley $s$-wave
exciton is found to have the lowest energy for $\Delta>0$; we henceforth
specialise to this state. Let us start from $T$=0. Fig. 3a shows the exciton
binding energy vs the gap parameter $\Delta$ for $\epsilon_{r}=1$ and three
values of distance to the gate $d=\\{20,100,1000\\}$nm; the solid and dashed
lines are computed from the BSE (6) and LSE (7), respectively, whereby only
the BSE includes retardation effects. First, we see that retardation
significantly influences the binding energy — one could think of retardation
of the screened potential acting as an effective dynamic boson mode which is
enhancing the binding of electrons and holes. Second, the plot marks a
condensation region, whereby $\epsilon_{b}\geq 2\Delta$. At $\epsilon_{r}=1$,
condensation occurs for gates placed beyond a critical distance
$d>d_{c}\approx 10-15$ nm. In particular, for $d=\\{20,100,1000\\}$nm the
quantum critical point is $\Delta_{c}=\\{31,55,62\\}$meV. Considering instead
hBN encapsulation, such that $\epsilon_{r}=3.9$, and taking $d=100$ nm, the
critical point is reduced to $\Delta_{c}=3$ meV. These results highlight the
strong influence of gate and dielectric screening on the excitonic binding
energy and condensation transition.

To establish the phase boundary of Fig. 3(b), we solve
$\epsilon_{b}(T_{c})=2\Delta$. We note that for $\Delta<\Delta_{c}$, the two-
particle problem makes sense only at or above the critical temperature, where
$T_{c}$ is understood to be a BKT transition temperature. For $\epsilon_{r}=1$
and $d=\\{100,1000\\}$nm maximum critical temperatures are significant,
$T_{c}\approx\\{100,115\\}$K, respectively. For $\epsilon_{r}=3.9$, $d=100$nm
the superfluid dome is comparatively small. Fig. 3(b) shows that as $\Delta\to
0$, the $T_{c}(\Delta)$ is rapidly decreasing. However, we stress that we do
not propagate the BSE technique down to exactly $\Delta=0$; here the system
becomes semi-metallic and the two-body technique employed here becomes
prohibitively expensive numerically. Physically, the strong suppression of
$T_{c}$ at small band gap $\Delta$ is due to the enhanced thermal excitation
to the conduction band; the thermally excited states act as a source of
metallic screening, and thereby have a significant affect on the Coulomb
attraction. This feature, i.e. strongly enhanced screening, makes this problem
highly non-BCS; BCS provides a simple relation between $T=0$ order parameter
and $T_{c}$, which derives from the Pauli blocking factor (i.e. thermal
occupation factors). In our case, we have both (thermal) Pauli blocking as
well as thermal screening of the interaction. We find that the thermal
screening plays the dominant role in melting the order, and thus we do not
recover the standard BCS relation.

Finally, we mention that within the BSE two-body formalism we arrive at the
following critical scaling of $T_{c}$ near $\Delta\to\Delta_{c}^{-}$ (see
Supplement [41]),

$\displaystyle
T_{c}(\Delta)\sim\Delta_{c}/\ln\left(\frac{g\Delta_{c}}{\Delta_{c}-\Delta}\right)$
(8)

where $g$ is a dimensionless combination of the interaction strength and
density of states, as well as other dimensionless numerical factors that we do
not evaluate.

Condensate phase — Entering the condensate phase we can no longer apply two-
body techniques, and therefore do not expect to make quantitative predictions.
Even so, the results below provide crucial predictions for future experimental
tests of the exciton condensate. The discussion from here on will be based on
a mean-field Hamiltonian,
$H_{\text{MF}}=\sum_{\bm{p},\tau,s}c^{\dagger}_{\bm{p},\tau,s}\omega_{\bm{p}}c_{\bm{p},\tau,s}+v^{\dagger}_{\bm{p},\tau,s}(-\omega_{\bm{p}})v_{\bm{p},\tau,s}+\sum_{\bm{p},\tau,\tau^{\prime},s,s^{\prime}}c^{\dagger}_{\bm{p},\tau,s}\Phi(p)_{\tau,\tau,s,s^{\prime}}v_{\bm{p},\tau^{\prime},s^{\prime}}$
+h.c., with $c^{\dagger}_{\bm{p},\tau,s}(v^{\dagger}_{\bm{p},\tau,s})$ the
creation operators for conduction (valence) electrons, and with
$\Phi(p)_{\tau,\tau,s,s^{\prime}}$ the order parameter. Explicitly for the
$s$-wave intervalley excitonic condensate, it takes the form

$\displaystyle\Phi(p)_{\tau,\tau^{\prime},s,s^{\prime}}=\Phi_{0}(p)(\tau_{x})_{\tau,\tau^{\prime}}e^{i\frac{(\tau-\tau^{\prime})}{2}\phi}(d_{\mu}s_{\mu}is_{y})_{s,s^{\prime}},$
(9)

where $\Phi_{0}(p)$ is the amplitude; $\phi$ an arbitrary phase encoding the
phase difference between the two distinct intervalley states, i.e. those with
valley indices $(\tau,\tau^{\prime})=(+-)$ and $(-+)$; $\tau_{x}$ is a Pauli
matrix acting on valley indices; $s_{\mu}$ are Pauli spin matrices and
$d_{\mu}$ are the components of a unit four-vector, with $\mu=0$ corresponding
to a spin-singlet and $\mu=1,2,3$ to the components of the spin-triplet –
there is an SO(4) degeneracy of this spin ordering vector $(d_{\mu})$.

We pause to note that a spontaneous symmetry breaking in unbiased bilayer
graphene has been considered [15]. Specifically, that work predicts a
spontaneous ferroelectric polarisation perpendicular to the plane, breaking a
$\mathbb{Z}_{2}$ ‘layer’ symmetry. By contrast, in the present work the
applied electric bias explicitly breaks the $\mathbb{Z}_{2}$ symmetry and
drives a single particle band gap ($2\Delta$). Our key prediction is the
spontaneous breakdown of a U(1) symmetry, and hence the onset of
superfluidity. We stress that the superfluid order parameter [Eq. (9)] is not
a ferroelectric; it does not couple linearly to an external electric field.
However, it does influence the layer polarization in a measurable way and this
opens a unique way to detect the superfluid quantum phase transition. We
discuss these details next.

Within the ordered phase $\Delta<\Delta_{c}$, and at $T=0$, we appeal to the
BCS/Eliashberg gap equation to estimate $\Phi_{0}(p)$ [for simplicity we, for
now, ignore the momentum dependence, such that $\Phi_{0}(p)=\Phi_{0}$]. Near
the critical point $\Delta\to\Delta_{c}$, and to logarithmic accuracy, the gap
equation gives
$1=g^{\prime}\ln(\Lambda^{\prime}/\sqrt{\Delta^{2}+|\Phi_{0}|^{2}})$ or
$|\Phi_{0}|=\text{Re}\sqrt{\Delta_{c}^{2}-\Delta^{2}}$. Here
$\Delta_{c}=\Lambda^{\prime}e^{-1/g^{\prime}}$, $g^{\prime}\sim g$ is a
dimensionless combination of the interaction and density of states and
$\Lambda^{\prime}\sim\Lambda$ is a UV cut-off.

The natural, measurable quantity of the system is not $\Phi_{0}$, but instead
the layer polarization. In the absence of excitonic order, $\Phi_{0}=0$, the
polarization is ${\cal P}_{\Delta}$ of Eq. (4). Polarization results from the
valence electrons belonging predominantly to, say, the bottom layer. Since
$\Phi_{0}\neq 0$ ultimately corresponds to removing valence electrons (bottom
layer) and enhancing conduction electrons (top layer), one expects
$\Phi_{0}\neq 0$ to reduce the ground state polarization. The polarization in
the condensate phase is then simply ${\cal
P}_{\Delta,\Phi_{0}}=\frac{2m\Delta}{\pi}\ln\left(\Lambda/\sqrt{\Delta^{2}+|\Phi_{0}|^{2}}\right)$.
We see that $\Phi_{0}\neq 0$ indeed reduces the layer polarization, with
leading correction quadratic in $\Phi_{0}$. Further, we appeal to the gap
equation solution to arrive a key prediction,

$\displaystyle{\cal P}_{\Delta,\Phi_{0}}-{\cal
P}_{\Delta}=-\frac{m|\Phi_{0}|^{2}}{\pi\Delta}\xrightarrow[\Delta\to\Delta_{c}^{-}]{}-\frac{2m(\Delta_{c}-\Delta)}{\pi}.$
(10)

This expression shows that there is a linear in $\Delta$ (and hence in
external electric field $E$) reduction of the layer polarization in the
vicinity of the critical point. We anticipate the polarization being readily
measured via quantum capacitance, see e.g. [43, 44, 45], which would provide a
direct experimental test of the exciton condensation transition.

Figure 4: Dimensionless layer polarization, as a function of band gap
parameter $\Delta$, with and without the condensate – corresponding to ${\cal
P}_{\Delta,\Phi}$ and ${\cal P}_{\Delta}$ [Eq. (4)] , respectively. Here we
use the meanfield expression
$|\Phi_{0}|=\text{Re}\sqrt{\Delta_{c}^{2}-\Delta^{2}}$, at $T=0$, with
$\Delta_{c}=62$ meV corresponding to system parameters $\epsilon_{r}=1,d=1000$
nm. Due to the onset of the condensate $\Phi$, a cusp is seen in reduced layer
polarization, ${\cal P}_{\Delta,\Phi}/(m\Delta)$, at the critical point
$\Delta_{c}$.

Finally, we turn to the in-plane real-space structure of the condensate. The
intervalley ordering [Eq. (9)] implies that, in real-space, the condensate
exhibits a pair density wave pattern, denoting
$\tilde{\Phi}_{s,s^{\prime}}(\bm{r},\bm{r}^{\prime})=\tilde{\Phi}(\bm{r},\bm{r}^{\prime})(d_{\mu}s_{\mu}is_{y})_{s,s^{\prime}}$,

$\displaystyle\tilde{\Phi}(\bm{r},\bm{r}^{\prime})$
$\displaystyle=\tilde{\Phi}_{0}(\bm{r}-\bm{r}^{\prime})\cos(\bm{K}\cdot(\bm{r}+\bm{r}^{\prime})+\phi)$
(11)

with $\Phi_{0}(\bm{r}-\bm{r}^{\prime})$ a complex amplitude (derived in the
Supplement [41]). Fluctuations of $\phi$ correspond to gapless sliding modes
of the pair density wave. The pair density wave is spatially modulated with a
periodicity of three unit cells. In the context of superconductivity, such a
density wave pattern has been analysed at length in Ref. [46] and has been
linked to higher-order topology in Refs. [47, 48] – we leave the question of
higher topology of the mean-field order Eq. (11) for future work.

Discussion — We presented a description for the excitonic bound states in
biased BLG, and their subsequent condensation. Owing to biased BLG being a
band insulator, we were able to apply two-body techniques. This is contrasted
to exciton formation and condensation from a semi-metallic ground state,
whereby many-body techniques are necessary, and resultantly uncontrolled
approximations are often required.

We examined suspended biased BLG, with gates $d>d_{c}\approx 10-15$ nm. In
this regime/setup, we show that excitons condense for $\Delta<\Delta_{c}$,
Fig. 3(a). This is our primary theoretical finding. For distance to gates
$d=100$nm (1000nm) the maximum critical temperature is predicted to be $100$K
($115$K), and is optimised at non-zero bias, Fig. 3(b). Entering the BEC
phase, our two-body techniques are no longer quantitatively accurate, however,
we are able to establish several qualitative features of the condensate.
Experimentally distinguishing between the excitonic condensate/insulator and
the band insulator is a challenging task. To this end, we found that the
excitons influence the macroscopic dipole moment [c.f. (10)], and proposed a
smoking gun test for exciton condensation – quantum capacitance measurements
of the dipole moment as a function of the bias field, which tunes the band
insulator to exciton condensate phase transition [c.f. Fig. 4]. We hope to
entice direct experimental searches for the excitonic condensate in biased
bilayer graphene. Future theoretical work may consider other candidate
materials using the two-body approach applied here.

###### Acknowledgements.

We thank Dmitry Efimkon, Alex Hamilton, Mike Zhitomirsky and Jörg Schmalian
for enlightening conversations. We acknowledge funding support from the
Australian Research Council Centre of Excellence in Future Low-Energy
Electronics Technology (FLEET) (CE170100039).

## References

  * [1] Y. E. Lozovik and V. I. Yudson, Feasibility of superfluidity of paired spatially separated electrons and holes; a new superconductivity mechanism, JETP Lett. (USSR) (Engl. Transl.); (United States) .
  * Pogrebinskii [1977] M. Pogrebinskii, Mutual drag of carriers in a semiconductor-insulator-semiconductor system, Soviet Physics-Semiconductors 11, 372 (1977).
  * Blatt  _et al._ [1962] J. M. Blatt, K. Böer, and W. Brandt, Bose-einstein condensation of excitons, Physical Review 126, 1691 (1962).
  * Kellogg  _et al._ [2004a] M. Kellogg, J. Eisenstein, L. Pfeiffer, and K. West, Vanishing hall resistance at high magnetic field in a double-layer two-dimensional electron system, Physical review letters 93, 036801 (2004a).
  * Su and MacDonald [2008] J.-J. Su and A. MacDonald, How to make a bilayer exciton condensate flow, Nature Physics 4, 799 (2008).
  * Wang  _et al._ [2019] R. Wang, O. Erten, B. Wang, and D. Y. Xing, Prediction of a topological p + ip excitonic insulator with parity anomaly, Nature Communications 10, 210 (2019).
  * Varsano  _et al._ [2020] D. Varsano, M. Palummo, E. Molinari, and M. Rontani, A monolayer transition-metal dichalcogenide as a topological excitonic insulator, Nature Nanotechnology 15, 367 (2020).
  * Perfetto and Stefanucci [2020] E. Perfetto and G. Stefanucci, Floquet topological phase of nondriven $p$-wave nonequilibrium excitonic insulators, Phys. Rev. Lett. 125, 106401 (2020).
  * Sun and Millis [2021] Z. Sun and A. J. Millis, Topological charge pumping in excitonic insulators, Phys. Rev. Lett. 126, 027601 (2021).
  * Liu  _et al._ [2021] Z.-R. Liu, L.-H. Hu, C.-Z. Chen, B. Zhou, and D.-H. Xu, Topological excitonic corner states and nodal phase in bilayer quantum spin hall insulators, Phys. Rev. B 103, L201115 (2021).
  * Scammell  _et al._ [2022] H. D. Scammell, J. Ingham, T. Li, and O. P. Sushkov, Chiral excitonic order, quantum anomalous hall effect, and superconductivity from twofold van hove singularities in kagome metals (2022), arXiv:2201.02643 [cond-mat.supr-con] .
  * Keldysh and Kopaev [1964] L. V. Keldysh and Y. V. Kopaev, Possible instability of the semimetallic state against coulomb interaction, Fiz. Tverd. Tela., 6, 2791 (1964) [Sov. Phys. Solid State 6, 2219 (1965)] (1964).
  * Jérome  _et al._ [1967] D. Jérome, T. M. Rice, and W. Kohn, Excitonic insulator, Phys. Rev. 158, 462 (1967).
  * Halperin and Rice [1968] B. Halperin and T. Rice, The excitonic state at the semiconductor-semimetal transition (Academic Press, 1968) pp. 115–192.
  * Nandkishore and Levitov [2010] R. Nandkishore and L. Levitov, Dynamical screening and excitonic instability in bilayer graphene, Phys. Rev. Lett. 104, 156803 (2010).
  * Song  _et al._ [2012] K. W. Song, Y.-C. Liang, and S. Haas, Excitonic instabilities and insulating states in bilayer graphene, Phys. Rev. B 86, 205418 (2012).
  * Apinyan and Kopeć [2016] V. Apinyan and T. K. Kopeć, Excitonic gap formation and condensation in the bilayer graphene structure, Physica Scripta 91, 095801 (2016).
  * Akzyanov  _et al._ [2014] R. S. Akzyanov, A. O. Sboychakov, A. V. Rozhkov, A. L. Rakhmanov, and F. Nori, $aa$-stacked bilayer graphene in an applied electric field: Tunable antiferromagnetism and coexisting exciton order parameter, Phys. Rev. B 90, 155415 (2014).
  * Apinyan and Kopeć [2021] V. Apinyan and T. K. Kopeć, Antiferromagnetic ordering and excitonic pairing in aa-stacked bilayer graphene, Phys. Rev. B 104, 075426 (2021).
  * Min  _et al._ [2008] H. Min, R. Bistritzer, J.-J. Su, and A. H. MacDonald, Room-temperature superfluidity in graphene bilayers, Phys. Rev. B 78, 121401 (2008).
  * Lozovik and Sokolik [2008] Y. E. Lozovik and A. A. Sokolik, Electron-hole pair condensation in a graphene bilayer, JETP Letters 87, 55 (2008).
  * Zhang and Joglekar [2008] C.-H. Zhang and Y. N. Joglekar, Excitonic condensation of massless fermions in graphene bilayers, Phys. Rev. B 77, 233405 (2008).
  * Kharitonov and Efetov [2008] M. Y. Kharitonov and K. B. Efetov, Electron screening and excitonic condensation in double-layer graphene systems, Phys. Rev. B 78, 241401 (2008).
  * Kharitonov and Efetov [2010] M. Y. Kharitonov and K. B. Efetov, Excitonic condensation in a double-layer graphene system, Semiconductor Science and Technology 25, 034004 (2010).
  * Perali  _et al._ [2013] A. Perali, D. Neilson, and A. R. Hamilton, High-temperature superfluidity in double-bilayer graphene, Phys. Rev. Lett. 110, 146803 (2013).
  * Su and MacDonald [2017] J.-J. Su and A. H. MacDonald, Spatially indirect exciton condensate phases in double bilayer graphene, Phys. Rev. B 95, 045416 (2017).
  * Efimkin  _et al._ [2020] D. K. Efimkin, G. W. Burg, E. Tutuc, and A. H. MacDonald, Tunneling and fluctuating electron-hole cooper pairs in double bilayer graphene, Phys. Rev. B 101, 035413 (2020).
  * Conti  _et al._ [2019] S. Conti, A. Perali, F. M. Peeters, and D. Neilson, Multicomponent screening and superfluidity in gapped electron-hole double bilayer graphene with realistic bands, Phys. Rev. B 99, 144517 (2019).
  * Burg  _et al._ [2018] G. W. Burg, N. Prasad, K. Kim, T. Taniguchi, K. Watanabe, A. H. MacDonald, L. F. Register, and E. Tutuc, Strongly enhanced tunneling at total charge neutrality in double-bilayer graphene-${\mathrm{wse}}_{2}$ heterostructures, Phys. Rev. Lett. 120, 177702 (2018).
  * Du  _et al._ [2017a] L. Du, X. Li, W. Lou, G. Sullivan, K. Chang, J. Kono, and R.-R. Du, Evidence for a topological excitonic insulator in inas/gasb bilayers, Nature Communications 8, 1971 (2017a).
  * Liu  _et al._ [2017] X. Liu, K. Watanabe, T. Taniguchi, B. I. Halperin, and P. Kim, Quantum hall drag of exciton condensate in graphene, Nature Physics 13, 746 (2017).
  * Li  _et al._ [2017] J. I. A. Li, T. Taniguchi, K. Watanabe, J. Hone, and C. R. Dean, Excitonic superfluid phase in double bilayer graphene, Nature Physics 13, 751 (2017).
  * Li  _et al._ [2019] J. I. A. Li, Q. Shi, Y. Zeng, K. Watanabe, T. Taniguchi, J. Hone, and C. R. Dean, Pairing states of composite fermions in double-layer graphene, Nature Physics 15, 898 (2019).
  * Kellogg  _et al._ [2004b] M. Kellogg, J. P. Eisenstein, L. N. Pfeiffer, and K. W. West, Vanishing hall resistance at high magnetic field in a double-layer two-dimensional electron system, Phys. Rev. Lett. 93, 036801 (2004b).
  * Tutuc  _et al._ [2004] E. Tutuc, M. Shayegan, and D. A. Huse, Counterflow measurements in strongly correlated gaas hole bilayers: Evidence for electron-hole pairing, Phys. Rev. Lett. 93, 036802 (2004).
  * Wiersma  _et al._ [2004] R. D. Wiersma, J. G. S. Lok, S. Kraus, W. Dietsche, K. von Klitzing, D. Schuh, M. Bichler, H.-P. Tranitz, and W. Wegscheider, Activated transport in the separate layers that form the ${\nu}_{T}=1$ exciton condensate, Phys. Rev. Lett. 93, 266805 (2004).
  * McCann and Koshino [2013] E. McCann and M. Koshino, The electronic properties of bilayer graphene, Reports on Progress in Physics 76, 056503 (2013).
  * McCann and Fal’ko [2006] E. McCann and V. I. Fal’ko, Landau-level degeneracy and quantum hall effect in a graphite bilayer, Phys. Rev. Lett. 96, 086805 (2006).
  * Zhang  _et al._ [2009] Y. Zhang, T.-T. Tang, C. Girit, Z. Hao, M. C. Martin, A. Zettl, M. F. Crommie, Y. R. Shen, and F. Wang, Direct observation of a widely tunable bandgap in bilayer graphene, Nature 459, 820 (2009).
  * Scammell and Sushkov [2022] H. D. Scammell and O. P. Sushkov, Dynamical screening and excitonic bound states in biased bilayer graphene (2022).
  * [41] See Supplemental Material for more details on the polarization operator, the form factors, the critical temperature scaling, and the real-space order parameter .
  * Berestetskii  _et al._ [1982] V. Berestetskii, E. Lifshitz, and L. Pitaevskii,  _Quantum Electrodynamics: Volume 4_, Course of theoretical physics (Elsevier Science, 1982).
  * Yang  _et al._ [1997] M. J. Yang, C. H. Yang, B. R. Bennett, and B. V. Shanabrook, Evidence of a hybridization gap in “semimetallic” InAs/GaSb systems, Phys. Rev. Lett. 78, 4613 (1997).
  * Du  _et al._ [2017b] L. Du, X. Li, W. Lou, G. Sullivan, K. Chang, J. Kono, and R.-R. Du, Evidence for a topological excitonic insulator in inas/gasb bilayers, Nature communications 8, 1 (2017b).
  * Saberi-Pouya  _et al._ [2020] S. Saberi-Pouya, S. Conti, A. Perali, A. F. Croxall, A. R. Hamilton, F. M. Peeters, and D. Neilson, Experimental conditions for the observation of electron-hole superfluidity in gaas heterostructures, Physical Review B 101, 140501 (2020).
  * Roy and Herbut [2010] B. Roy and I. F. Herbut, Unconventional superconductivity on honeycomb lattice: Theory of kekule order parameter, Phys. Rev. B 82, 035429 (2010).
  * Li  _et al._ [2020] T. Li, J. Ingham, and H. D. Scammell, Artificial graphene: Unconventional superconductivity in a honeycomb superlattice, Phys. Rev. Research 2, 043155 (2020).
  * Li  _et al._ [2021] T. Li, M. Geier, J. Ingham, and H. D. Scammell, Higher-order topological superconductivity from repulsive interactions in kagome and honeycomb systems, 2D Materials 9, 015031 (2021).

## Supplemental Material

##  I Hamiltonian and wavefunctions

The Hamiltonian and wavefunctions are (in zero field)

$\displaystyle H_{\tau}$ $\displaystyle=-\frac{(\tau p_{x}\mp
ip_{y})^{2}}{2m}\sigma_{\pm}+\Delta\sigma_{z}$
$\displaystyle|\psi^{(+)}_{\bm{p},\tau}\rangle$
$\displaystyle=\frac{1}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}}\begin{pmatrix}-1\\\
\frac{\varepsilon_{\bm{p}}-\Delta}{p^{2}/(2m)}e^{2i\tau\theta_{\bm{p}}}\end{pmatrix},$
$\displaystyle|\psi^{(-)}_{\bm{p},\tau}\rangle={\cal
P}|\psi^{(+)}_{\bm{p},\tau}\rangle$
$\displaystyle=\frac{1}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}}\begin{pmatrix}\frac{\varepsilon_{\bm{p}}-\Delta}{p^{2}/(2m)}e^{-2i\tau\theta_{\bm{p}}}\\\
1\end{pmatrix},$ $\displaystyle{\cal P}$ $\displaystyle=i\sigma_{y}{\cal C}.$
(S1)

The operator ${\cal P}$ generates the particle-hole transformation, which is
an (anti)symmetry of this Hamiltonian. Defining $\psi_{-}$ in this way
correctly generates the phase/winding factors.

##  II Polarization Operator

The expression for the polarization generically takes the form,

$\displaystyle\Pi({\bm{q}},i\xi,T)$
$\displaystyle=4\sum_{\mu,\nu=\pm}\int\frac{d^{2}p}{(2\pi)^{2}}\frac{\left(f_{\epsilon^{\mu}_{\bm{p}}}-f_{\epsilon^{\nu}_{\bm{p+q}}}\right)}{i\xi+\epsilon^{\mu}_{\bm{p}}-\epsilon^{\nu}_{\bm{p+q}}}F^{\mu\nu}_{\bm{p,p+q}}.$
(S2)

Here the form factors are
$F^{\mu\nu}_{\bm{p,p+q}}=|\langle\psi^{(\mu)}_{\bm{p+q},\tau}|\psi^{(\nu)}_{\bm{p},\tau}\rangle|^{2}$,
with $\mu,\nu=\pm$. Using that
$-\epsilon^{-}_{\bm{p}}=\epsilon^{+}_{\bm{p}}\equiv\epsilon_{\bm{p}}$ and
$F^{--}_{\bm{p,p+q}}=F^{++}_{\bm{p,p+q}}$,
$F^{-+}_{\bm{p,p+q}}=F^{+-}_{\bm{p,p+q}}$, we get

$\displaystyle\Pi({\bm{q}},i\xi,T)$
$\displaystyle=8\int\frac{d^{2}p}{(2\pi)^{2}}\left[\frac{(\epsilon_{\bm{p}}-\epsilon_{\bm{p+q}})\left(f_{\epsilon_{\bm{p}}}-f_{\epsilon_{\bm{p+q}}}\right)}{\xi^{2}+(\epsilon_{\bm{p}}-\epsilon_{\bm{p+q}})^{2}}F^{++}_{\bm{p,p+q}}+\frac{(\epsilon_{\bm{p}}+\epsilon_{\bm{p+q}})\left(f_{\epsilon_{\bm{p}}}+f_{\epsilon_{\bm{p+q}}}-1\right)}{\xi^{2}+(\epsilon_{\bm{p}}+\epsilon_{\bm{p+q}})^{2}}F^{+-}_{\bm{p,p+q}}\right].$
(S3)

##  III Vertex Form Factors

The Coulomb interaction is taken to be

$\displaystyle H_{int}$
$\displaystyle=\sum_{\bm{p}_{1},\bm{p}_{2},\bm{p}_{3}}\sum_{\tau,\tau^{\prime}}V(\bm{p}_{2}-\bm{p}_{1})\Psi^{\dagger}_{\bm{p}_{1},\tau}\Psi^{\dagger}_{\bm{p}_{2},\tau^{\prime}}\Psi_{\bm{p}_{3},\tau^{\prime}}\Psi_{\bm{p}_{1}+\bm{p}_{2}-\bm{p}_{3},\tau},$
(S4)

which neglects valley-exchange. We transform to band basis,

$\displaystyle\Psi_{\bm{p},\tau}$ $\displaystyle={\cal
U}_{\bm{p},\tau}\begin{pmatrix}c_{\bm{p},\tau}\\\
v_{\bm{p},\tau}\end{pmatrix},\ {\cal
U}_{\bm{p},\tau}=(|\psi^{(+)}_{\bm{p},\tau}\rangle,|\psi^{(-)}_{\bm{p},\tau}\rangle),$

with $c_{\bm{p},\tau}$ and $v_{\bm{p},\tau}$ the destruction operators for
conduction and valence electrons. Restricting consideration to the exciton
channel, denoted $H_{int}^{X}$, the interaction becomes,

$\displaystyle H_{int}^{X}$
$\displaystyle=\sum_{\bm{p}_{1},\bm{p}_{2}}\sum_{\tau,\tau^{\prime}}V(\bm{p}_{2}-\bm{p}_{1})c^{\dagger}_{\bm{p}_{1},\tau}c_{\bm{p}_{2},\tau}v^{\dagger}_{\bm{p}_{2},\tau^{\prime}}v_{\bm{p}_{1},\tau^{\prime}}\langle\psi^{(-)}_{\bm{p}_{2},\tau^{\prime}}|\psi^{(-)}_{\bm{p}_{1},\tau^{\prime}}\rangle\langle\psi^{(+)}_{\bm{p}_{1},\tau}|\psi^{(+)}_{\bm{p}_{2},\tau}\rangle.$
(S5)

In the main text we denote the from factor
$Z_{\bm{p}_{1},\bm{p}_{2}}^{\tau^{\prime},\tau}=\langle\psi^{(-)}_{\bm{p}_{2},\tau^{\prime}}|\psi^{(-)}_{\bm{p}_{1},\tau^{\prime}}\rangle\langle\psi^{(+)}_{\bm{p}_{1},\tau}|\psi^{(+)}_{\bm{p}_{2},\tau}\rangle$.

##  IV Critical temperature scaling

Setting $E=0$, the $T=0$ the Lippmann-Schwinger equation gives the condition
for $\Delta_{c}$

$\displaystyle 2\sqrt{\frac{p^{4}}{4m^{2}}+\Delta_{c}^{2}}\Psi_{\bm{p}}$
$\displaystyle=\int\frac{d^{2}p^{\prime}}{(2\pi)^{2}}V^{0}_{\bm{p},\bm{p}^{\prime}}\Psi_{\bm{p}^{\prime}}.$
(S6)

At $T>0$, the condition $E=0$ requires $\Delta<\Delta_{c}$, and is given by

$\displaystyle 2\sqrt{\frac{p^{4}}{4m^{2}}+\Delta_{c}^{2}}\Psi_{\bm{p}}$
$\displaystyle=\int\frac{d^{2}p^{\prime}}{(2\pi)^{2}}\left[V^{0}_{\bm{p},\bm{p}^{\prime}}+\delta
V_{\bm{p},\bm{p}^{\prime}}e^{-\frac{\Delta}{T_{c}}}\right]\Psi_{\bm{p}^{\prime}}.$
(S7)

Using that the characteristic momentum is $p=\sqrt{2m\Delta}$, we combine
these two equations as

$\displaystyle\Delta_{c}$ $\displaystyle=\nu_{0},$ $\displaystyle\Delta$
$\displaystyle=\left(\nu_{0}-\nu_{1}e^{-\frac{\Delta}{T_{c}}}\right),$
$\displaystyle(\Delta_{c}-\Delta)$
$\displaystyle=\nu_{1}e^{-\frac{\Delta}{T_{c}}},\to
T_{c}=\Delta/\ln\left(\frac{\nu_{1}}{\Delta_{c}-\Delta}\right).$ (S8)

##  V Real-space order parameter

In this appendix we derive the effective lattice model for the dominant
excitonic order parameter. Introducing the real-space creation operators for
the conduction and valence bands via,

$\displaystyle c_{\bm{p},\tau,s}^{\dagger}$
$\displaystyle=\sum_{\bm{r}}\phi^{(+)}_{\bm{p},\tau}(\bm{r})\tilde{c}^{\dagger}_{\bm{r},s},$
$\displaystyle v_{\bm{p},\tau,s}^{\dagger}$
$\displaystyle=\sum_{\bm{r}}\phi^{(-)}_{\bm{p},\tau}(\bm{r})\tilde{v}^{\dagger}_{\bm{r},s}.$
(S9)

With $\bm{r}$ spanning the real-space lattice sites, and comprising two
sublattices, $\sigma=A_{1},B_{2}$, and $\\{a(\bm{r}\in A)=1,a(\bm{r}\in
B)=0\\}$ and $\\{b(\bm{r}\in A)=0,b(\bm{r}\in B)=1\\}$. Here,

$\displaystyle\phi^{(+)}_{\bm{p},\tau}(\bm{r})$
$\displaystyle=e^{i(\bm{p}+\tau\bm{K})}\left(\alpha^{(+)}_{\tau,\bm{k}}a(\bm{r})+\beta^{(+)}_{\tau,\bm{k}}b(\bm{r})\right),$
$\displaystyle\phi^{(-)}_{\bm{p},\tau}(\bm{r})$
$\displaystyle=e^{i(\bm{p}+\tau\bm{K})}\left(\alpha^{(-)}_{\tau,\bm{k}}a(\bm{r})+\beta^{(-)}_{\tau,\bm{k}}b(\bm{r})\right),$
(S10)

with definitions,

$\displaystyle\alpha^{(+)}_{\tau,\bm{p}}$
$\displaystyle=-\cos\gamma_{\bm{p}}=\frac{-1}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}},$
$\displaystyle\beta^{(+)}_{\tau,\bm{p}}$
$\displaystyle=\sin\gamma_{\bm{p}}e^{2i\tau\theta_{\bm{p}}}=\frac{\varepsilon_{\bm{p}}-\Delta}{p^{2}/(2m)}\frac{e^{2i\tau\theta_{\bm{p}}}}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}},$
$\displaystyle\alpha^{(-)}_{\tau,\bm{p}}$
$\displaystyle=\sin\gamma_{\bm{p}}e^{-2i\tau\theta_{\bm{p}}}=\frac{\varepsilon_{\bm{p}}-\Delta}{p^{2}/(2m)}\frac{e^{-2i\tau\theta_{\bm{p}}}}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}},$
$\displaystyle\beta^{(-)}_{\tau,\bm{p}}$
$\displaystyle=\cos\gamma_{\bm{p}}=\frac{1}{\sqrt{\frac{(\varepsilon_{\bm{p}}-\Delta)^{2}}{p^{4}/(4m^{2})}+1}}.$
(S11)

The mean-field Hamiltonian becomes,

$\displaystyle H_{\Phi}$
$\displaystyle=\sum_{\bm{p},\tau,\tau^{\prime},s,s^{\prime}}c_{\bm{p},\tau,s}^{\dagger}\Phi(p)_{\tau,\tau^{\prime},s,s^{\prime}}v_{\bm{p},\tau^{\prime},s^{\prime}}=\sum_{\bm{p},\tau,\tau^{\prime},s,s^{\prime}}\sum_{\bm{r},\bm{r}^{\prime}}\phi^{(+)}_{\bm{p},\tau}(\bm{r})(\phi^{(-)}_{\bm{p},\tau^{\prime}})^{*}(\bm{r}^{\prime})\Phi(p)_{\tau,\tau^{\prime},s,s^{\prime}}\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}+\text{h.c.}\
.$ (S12)

We consider intervalley pairing in the $s$-wave channel and with arbitrary
spin ordering (singlet or triplet). There is both a symmetric and an anti-
symmetric combination of the valleys,

$\displaystyle\Phi(p)_{\tau,\tau^{\prime},s,s^{\prime}}$
$\displaystyle=\begin{cases}\Phi_{0}(p)(\tau_{x})_{\tau,\tau^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}\\\
\Phi_{0}(p)(i\tau_{y})_{\tau,\tau^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}.\end{cases}$
(S13)

In fact, there is a U(1) rotational symmetry that connects these distinct
valley structures – we more compactly write

$\displaystyle\Phi(p)_{\tau,\tau^{\prime},s,s^{\prime}}$
$\displaystyle=\Phi_{0}(p)(\tau_{x})_{\tau,\tau^{\prime}}e^{i\frac{(\tau-\tau^{\prime})}{2}\phi}(d_{\mu}s_{\mu})_{s,s^{\prime}}$
(S14)

Let us consider the case that $\bm{r}\in A$ and $\bm{r}^{\prime}\in B$,

$\displaystyle H^{AB}_{\Phi}$
$\displaystyle=\sum_{\bm{p},\tau,s,s^{\prime}}\sum_{\bm{r}\in
A,\bm{r}^{\prime}\in
B}\left(e^{i(\bm{p}+\tau\bm{K})\cdot\bm{r}}\alpha^{(+)}_{\tau,\bm{p}}a(\bm{r})\right)\left(e^{-i(\bm{p}-\tau\bm{K})\cdot\bm{r}^{\prime}}\beta^{(-)}_{-\tau,\bm{p}}b(\bm{r}^{\prime})\right)\Phi_{0}(p)e^{i\tau\phi}\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}+\text{h.c.}\
$ $\displaystyle=\sum_{\bm{p},\tau,s,s^{\prime}}\sum_{\bm{r}\in
A,\bm{r}^{\prime}\in
B}\left[e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}e^{i\tau\bm{K}\cdot(\bm{r}+\bm{r}^{\prime})}\alpha^{(+)}_{\tau,\bm{p}}\beta^{(-)}_{-\tau,\bm{p}}\Phi_{0}(p)e^{i\tau\phi}\right]\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}+\text{h.c.}\
,$ $\displaystyle=\sum_{\bm{r}\in A,\bm{r}^{\prime}\in
B}\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(-\cos^{2}\gamma_{\bm{p}})\Phi_{0}(p)\right]\left[\cos(\bm{K}\cdot(\bm{r}+\bm{r}^{\prime})+\phi)\right]\sum_{s,s^{\prime}}\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}+\text{h.c.}\
,$ $\displaystyle=\sum_{\bm{r}\in A,\bm{r}^{\prime}\in
B}\Gamma^{AB}(\bm{r}-\bm{r}^{\prime})\left[\cos(\bm{K}\cdot(\bm{r}+\bm{r}^{\prime})+\phi)\right]\sum_{s,s^{\prime}}\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}+\text{h.c.}\
.$ (S15)

We have defined the function,

$\displaystyle\Gamma^{AB}(\bm{r}-\bm{r}^{\prime})$
$\displaystyle\equiv\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(-\cos^{2}\gamma_{\bm{p}})\Phi_{0}(p)\right],$
(S16)

which determines the real-space amplitude and can be directly evaluated
numerically.

Performing the same procedure for all combinations of $\bm{r},\bm{r}^{\prime}$
in $\\{A_{1},B_{2}\\}$, we arrive at the expression,

$\displaystyle H_{\Phi}$
$\displaystyle=\sum_{\bm{r},\bm{r}}\Gamma(\bm{r}-\bm{r}^{\prime})\left[\cos(\bm{K}\cdot(\bm{r}+\bm{r}^{\prime})+\phi)\right]\sum_{s,s^{\prime}}\tilde{c}^{\dagger}_{\bm{r},s}\tilde{v}_{\bm{r}^{\prime},s^{\prime}}(d_{\mu}s_{\mu})_{s,s^{\prime}}+\text{h.c.},$
(S17)

with amplitude function,

$\displaystyle\Gamma(\bm{r}-\bm{r}^{\prime})$
$\displaystyle=\begin{cases}\Gamma^{AA}(\bm{r}-\bm{r}^{\prime})=\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(-\cos\gamma_{\bm{p}}\sin\gamma_{\bm{p}}e^{\pm
2i\tau\theta_{\bm{p}}})\Phi_{0}(p)\right],\ \text{for}\ \\{\bm{r}\in
A_{1},\bm{r}^{\prime}\in A_{1}\\}\\\
\Gamma^{BB}(\bm{r}-\bm{r}^{\prime})=\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(\cos\gamma_{\bm{p}}\sin\gamma_{\bm{p}}e^{\pm
2i\tau\theta_{\bm{p}}})\Phi_{0}(p)\right],\ \text{for}\ \\{\bm{r}\in
B_{2},\bm{r}^{\prime}\in B_{2}\\}\\\
\Gamma^{AB}(\bm{r}-\bm{r}^{\prime})=\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(-\cos^{2}\gamma_{\bm{p}})\Phi_{0}(p)\right],\
\text{for}\ \\{\bm{r}\in A_{1},\bm{r}^{\prime}\in B_{2}\\}\\\
\Gamma^{BA}(\bm{r}-\bm{r}^{\prime})=\sum_{\bm{p}}\left[2e^{i\bm{p}\cdot(\bm{r}-\bm{r}^{\prime})}(\sin^{2}\gamma_{\bm{p}}e^{\pm
4i\theta_{\bm{p}}})\Phi_{0}(p)\right],\ \text{for}\ \\{\bm{r}\in
B_{2},\bm{r}^{\prime}\in A_{1}\\}.\end{cases}$ (S18)