# BibleTTS: a large, high-fidelity, multilingual, and uniquely African speech
corpus

###### Abstract

BibleTTS is a large, high-quality, open speech dataset for ten languages
spoken in Sub-Saharan Africa. The corpus contains up to 86 hours of aligned,
studio quality 48kHz single speaker recordings per language, enabling the
development of high-quality text-to-speech models. The ten languages
represented are: Akuapem Twi, Asante Twi, Chichewa, Ewe, Hausa, Kikuyu,
Lingala, Luganda, Luo, and Yoruba. This corpus is a derivative work of Bible
recordings made and released by the Open.Bible project from Biblica. We have
aligned, cleaned, and filtered the original recordings, and additionally hand-
checked a subset of the alignments for each language. We present results for
text-to-speech models with Coqui TTS. The data is released under a commercial-
friendly CC-BY-SA license.

Index Terms: TTS, text-to-speech, speech synthesis, speech corpora, speech
data

##  1 Introduction

The majority of the world's approximately 7,000 languages [1] do not have open
speech datasets, and even fewer have high-quality data with aligned text and
speech, which can be used for training text-to-speech (TTS) models. The
creation of benchmark datasets such as Librispeech [2], LibriTTS [3], and
LJSpeech [4] enabled significant advances through community development on
common resources, but these resources cover few languages, and most TTS
systems evaluate on English only.

Speech synthesis systems have received significant attention in recent years
due to the advances provided by deep learning. These advances enable TTS
models to achieve improved naturalness with respect to human speech [5, 6, 7],
and improved synthesized speech as driven adoption of virtual assistants [8,
9]. However, neural models often require a non-trivial amount of data for
training. This necessity leaves many language communities under-served in the
development of speech technologies [10], and it further results in researchers
not evaluating models on diverse linguistic phenomena.

In this work, we present the BibleTTS corpus, a high-quality aligned speech
corpus for ten African languages. This data enables further research and
resource creation for these languages and will allow researchers to create
meaningful benchmarks against non-English languages.

Creating high-quality aligned datasets typically requires tools not available
for most languages, hindering the creation of datasets for lower-resourced
languages. Specifically, forced alignment of speech and text typically
requires pre-trained acoustic models and grapheme-to-phoneme (G2P) models.
This process can be challenging and error-prone without high-quality
resources. We demonstrate that it is possible to force-align data without
access to any pre-trained models (acoustic or G2P), and still produce quality
output.

Additionally, recent corpora that significantly expand linguistic coverage for
TTS datasets are often not freely available [11, 12], contain less single-
speaker data [13], and/or have lower-quality recordings. BibleTTS stands out
in this regard as it is a large, high-fidelity corpus made of single-speaker
recordings. The corpus is released under an open CC-BY-SA license. Corpus
links and samples created with our TTS models can be accessed from the project
website111https://masakhane-io.github.io/bibleTTS/.

##  2 Related Work

We focus on related work for African languages in the following section.
Existing publicly available datasets are typically small. For Yorùbá these
include a 2.75 hour corpus [14, 15] and a 4 hour multi-speaker dataset [16].
TWB Gamayun kits [17] include a 6-hour single speaker high-quality Swahili
speech corpus optimized for TTS training. Earlier Yorùbá TTS efforts typically
used bespoke private data [18, 19, 20, 21, 22]. For isiXhosa, Sesotho,
Setswana and Afrikaans, multi-speaker corpora of approximately 2 hours each
have been developed for TTS [23, 24]. The CMU Wilderness dataset [11] includes
up to 20 hours of high-quality, single-speaker data for several African
languages, but it is not publicly available and the alignments can contain
noise. TTS systems research for African languages has comprised development
efforts in frameworks like Festival [25] or MaryTTS [26] for Yorùbá [27, 28],
Ibibio [29], Amharic [30], Fon [31], isiZulu [32], KiSwahili [33]. While many
of these systems used concatenative synthesis, in large part because the
available corpora were small, there have also been investigations into
statistical parametric speech synthesis for Ibibio [34]. Finally, there have
been efforts in related tasks, such as grapheme-to-phoneme research for Yorùbá
[35], intonation modeling [21], and numeral preprocessing [36].

##  3 Languages represented

Table 1 shows the languages in the BibleTTS corpus, with their language
families, the number of speakers[1] and the regions in Africa where they are
spoken. The corpus consists of ten languages from the three largest language
families in Africa (Niger-Congo, Afro-Asiatic and Nilo-Saharan) and four
regions of Africa. All of these languages are tonal and are spoken primarily
in sub-Saharan Africa.

###  3.1 Language Characteristics222https://omniglot.com/writing/<language-
name>.htm

Table 1: Language, classification and statistics. All language classifications
and numbers of speakers are from Ethnologue.

|  | African | No. of
---|---|---|---
Language | Classification | Region | speakers
Éwé [ewe] | Niger-Congo / Kwa | West | 5.5M
Hausa [hau] | Afro-Asiatic / Chadic | West | 77M
Kikuyu [kik] | Niger-Congo / Bantu | East | 8.2M
Lingala [lin] | Niger-Congo / Bantu | Central | 40M
Luganda [lug] | Niger-Congo / Bantu | East | 11M
Luo [luo] | Nilo-Saharan / Luo–Acholi | East | 5.3M
Chichewa [nya] | Niger-Congo / Bantu | South-East | 14M
Akuapem Twi [aka] | Niger-Congo / Akan | West | 626k
Asante Twi [aka] | Niger-Congo / Akan | West | 3.8M
Yorùbá [yor] | Niger-Congo / Volta-Niger | West | 46M

Éwé [ewe] uses 35 Latin letters excluding (c, j, q), with 12 additional
letters (, dz, , f, gb, , kp, ny, ŋ, , ts, ). Ewe has three tones, and they
are marked in text.

Hausa [hau] uses two different writing scripts: Ajami and Boko. The Boko
script is the most widely used and is based on the Latin alphabet with 44
letters. The alphabet excludes letters (p, q, v and x) and uses 12 additional
letters: , , , y, kw, w, gw, ky, y, gy, sh, ts. Hausa is tonal, but tones are
not represented in text.

Kikuyu [kik] uses Latin script with 27 letters excluding (f, l, p, s, v, x, y,
z), and including additional nine letters (ĩ, ũ, mb, nd, nj, ng, ng`,ny, th).
Kikuyu uses two tones (high and low) but they are not marked in text.

Lingala [lin] uses the Latin script with 40 letters excluding (j, q, x) and
including an additional 17 letters (, gb, kp, mb, mf, mp, mv, nd, ng, ngb, nk,
ns, nt, ny, nz, , ts). Lingala uses two tones (high and low), but they are not
marked in text.

Luganda [lug] uses 24 Latin letters excluding (h, q, x), and including
additional two letters (ŋ, ny). Luganda uses three tones, but they are not
marked in text.

Luo [luo] or Dholuo uses Latin script with 31 letters excluding the letters
(c, q, x, v, z), and additional letters (ch, dh, mb, nd, ng’, ng, ny, nj, th,
sh). Luo has four tones, but they are not marked in text.

Chichewa [nya] uses the Latin script with 31 letters excluding (q, x, y), and
including additional eight letters (ch, kh, ng, ŋ, ph, tch, th, ŵ). Chichewa
uses two tones (high and low) but they are not marked in text.

Akan [aka] is a language with multiple dialects (including Fante, Bono,
Asante, and Akuapem), and they are collectively known as Twi. In this study,
we focus on Asante and Akuapem which are mutually intelligible and share the
same alphabets (referred to herein as aka-Asante and aka-Akuapem). Twi uses 22
Latin letters excluding (c,j,q,v,x,z), and including two additional letters (,
).

Yorùbá [yor] uses 25 Latin letters without the letters (c, q, v, x and z) and
with additional letters (ẹ, gb, ṣ, ọ). Yorùbá is a tonal language with three
tones: low, middle, and high. These tones are represented by the grave (e.g.
``è ''), optional macron (e.g. ``ē'') and acute (e.g. ``é'') accents
respectively but the mid tone is usually ignored in writing.

##  4 Corpus creation

The BibleTTS corpus consists of high-quality audio released as 48kHz, 24-bit,
mono-channel FLAC files. Recordings for each language are under professional
quality, close-microphone conditions (i.e., without background noise or echo).
BibleTTS is unique among open speech corpora for the volume of data per
speaker and suitability for TTS. The corpus consists of ten languages which
are under-represented in today's voice technology landscape, both in academia
and in industry. We release train/dev/test splits for each language, where dev
is the Book of Ezra, test is Colossians, and train is all other books.

Figure 1: Distribution of the sample length per language. Samples longer than
30s and with fewer than 10 characters were removed, and outlier segments were
detected and discarded as described in Section 4.2. Lingala is a slight
outlier with the majority of segments between 10 and 20 seconds, while the
other five languages have segments centered at 5-10s each.

###  4.1 Alignment

The BibleTTS corpus contains audio recordings and text transcripts (i.e.
``Open Contemporary Bible'' translations) which were released by Biblica via
the Open.Bible project.444https://open.bible/resources The original audio
recordings were 48kHz, mono-channel WAV, typically one recording per chapter
of the Bible. Each chapter was up to 30 minutes long, which is too long for
most modeling tasks. Verses are a natural alternative, as the text already
contains verse boundaries. Aligning at the verse level creates more manageable
recording lengths of up to 30 seconds (see Figure 1) which are more likely to
be consistent across languages than segmentation on voice activity detection
or other alternatives.

Potential challenges in alignment include additional content in either the
speech or text, such as spoken titles and headings or text annotations, and
the availability of pre-trained acoustic models and grapheme-to-phoneme
mappings. We have employed various alignment techniques depending on the
availability of verse timestamps and resources in each language, and evaluated
a subset of the alignments with native speakers.

####  4.1.1 Verse timestamps

Three languages (aka-Akuapem, aka-Asante, and lin) were straightforwardly
segmented using verse-level timestamps released by the Open.Bible project. The
timestamps show the start time of every verse, as well as when the book and
chapter titles were spoken. With these timestamps, verses were isolated and
saved as individual audio files using sox. These alignment scripts can be
found on Github at coqui-ai/open-bible-scripts. 555https://github.com/coqui-
ai/open-bible-scripts

####  4.1.2 Forced alignment using pre-trained acoustic models

Forced alignment is the process of extracting timestamps given an audio and a
transcript pair, and requires either a pre-trained acoustic model or training
one from scratch. For Hausa (hau), we opted to use the Montreal Forced Aligner
(MFA) [37] for which there is a pre-trained Hausa model [38]. The code is
open-sourced on Github666https://github.com/alpoktem/bible2speechDB. The
process is as follows:

  1. 1.

Audio of each chapter of each book is downloaded together with their script in
the form of an XML file,

  2. 2.

XML script is parsed and converted into a plain normalized text file.
Normalization entails: (a) adding the chapter title at the beginning of the
script as "Sura <chapter-no>", (b) converting numbers into written form using
a dictionary prepared with Hausa linguists, and (c) adding a new line after
every sentence ending punctuation mark (e.g. .?!").

  3. 3.

A grapheme-to-phoneme (G2P) dictionary is created from the word list extracted
from the transcripts using the Hausa G2P model.

  4. 4.

Alignment is performed for each chapter using the audio and normalized script
with a beam length of 1000.

  5. 5.

The time-aligned TextGrid file is processed in parallel with the sentence-
segmented transcript to partition the chapter audio into sentence-level audio
chunks with their transcriptions.

####  4.1.3 Forced alignment from scratch

Two languages (ewe and yor) were aligned via forced alignment from scratch.
Using only the found audio and transcripts (i.e., without a pre-trained
acoustic model), an acoustic model was trained and the data aligned with the
Montreal Forced Aligner. Graphemes were used as a proxy for phonemes in place
of G2P data. The code used to generate alignments can be found in the coqui-
ai/open-bible-scripts repository.777https://github.com/coqui-ai/open-bible-
scripts

After forced alignment, we used regular expressions to pull out whole verses
which were aligned such that silence occurred both at the beginning and the
end of a verse. Segmenting out audio at the verse-level instead of splitting
on silence may allow downstream TTS models to capture higher-level prosody.

###  4.2 Outlier detection

Following the alignment stage, we detected and removed outliers using the
data-checker toolkit together with human judgments. The relevant code is open-
sourced on Github at coqui-ai/data-checker.888https://github.com/coqui-
ai/data-checker First, all segments longer than 30 seconds, or less than 10
characters in the aligned transcript, were removed. Then, the removal of
outliers was performed and fine-tuned for each language independently until
the major offending samples999”Major offending samples” was not explicitly
defined, but refers to samples labeled by a non-native speaker of these
languages as containing obvious mismatches between transcripts and speech.
were no longer encountered, as described below.

Every pair of <audio,transcript> was assigned an "outlier score", and the most
extreme outliers were removed. First, the ratio of transcript length
(characters) to audio length (seconds) was calculated for each sample. Then a
Gaussian distribution was estimated for all samples in a given language.
Lastly, the number of standard deviations from the mean was calculated for
each sample. Outliers were excluded if they existed more than N standard
deviations away from the mean, where N was fine-tuned per language with an
iterative human-in-the-loop approach, until minimal offending samples were
encountered. For most languages, it was sufficient to exclude samples more
than 3 standard deviations from the mean (or .2% of the data). However, yor
notably required more outliers removed to attain a quality dataset. The
resulting distribution of segment lengths per language is shown in Figure 1.

###  4.3 Human evaluation of alignment quality

We facilitated human evaluation of both the alignment and the output of the
TTS models. In total, we collected labels from 15 annotators (three per
language) for ewe, hau, lin, aka-Asante, and aka-Akuapem and an additional
five annotators for yor. To judge the quality of <audio,transcript> pairs from
our alignments, we randomly sampled 50 example pairings of aligned transcripts
and the corresponding audio clips across the train, dev, and test sets.
Annotators selected the one option that best described the quality of the
alignment:

  1. 1.

Audio contains EXTRA words not in the transcript

  2. 2.

Audio is MISSING words that are in the transcript

  3. 3.

Audio is MISSING words AND includes EXTRA words

  4. 4.

No missing or extra words

In cases where the labels corresponding to various annotators disagreed, we
took the majority vote label. In cases where the number of labels was spread
evenly among different choices, we noted these as "conflicting." Results of
human evaluation are shown in Table 3.

As discussed in Section 4.1, some languages (aka-Asante, aka-Akuapem, lin)
were segmented using existing verse-level timestamp files. Interestingly,
annotators labeled these languages as having a high percentage of samples
where the audio contains additional words not present in the aligned text.
Aligning from scratch (ewe, yor) produced a greater proportion of segments
with exact matches between speech and text than using forced alignment with a
pre-trained acoustic model (hau). However, it should be noted that
significantly more data was removed due to outliers for yor, and less data
overall was aligned with ewe, yor (see the statistics for unaligned vs.
aligned hours in Table 2).

Table 2: Corpus statistics. The corpus consists of data for ten languages, of
which six have been aligned and formatted for immediate use to train TTS
models.

| Unaligned | Unaligned | Aligned | Aligned
---|---|---|---|---
Language | Hours | Samples | Hours | Samples
Éwé | 100.1 | 1,167 | 86.8 | 24,957
Hausa | 103.2 | 1,189 | 86.6 | 40,603
Kikuyu | 90.6 | 1,189 | – | –
Lingala | 151.7 | 1,189 | 71.6 | 15,117
Luganda | 110.4 | 1,189 | – | –
Luo | 80.4 | 1,189 | – | –
Chichewa | 115.9 | 1,162 | – | –
Akuapem Twi | 75.7 | 1,189 | 67.1 | 28,238
Asante Twi | 82.6 | 1,189 | 74.9 | 29,021
Yorùbá | 93.6 | 1,189 | 33.3 | 10,228

##  5 TTS Models

To experimentally validate the quality of our dataset, we train the VITS end-
to-end speech synthesis model [7] with the sampling rate 22050 Hz in the six
aligned languages. The chosen languages are Akuapem Twi, Asante Twi, Éwé,
Hausa, Lingala, and Yorùbá. We chose VITS for its state-of-the-art naturalness
and also for its robust alignment mechanism [39]. The model takes characters
as input and does not require a phonemizer.

To accelerate training we use transfer learning. We start from a model pre-
trained on LJSpeech [40] for 1M steps, which is available via the Coqui TTS
repository101010https://github.com/coqui-ai/TTS. We continue training for
approximately 110K steps for each one of the languages. We use the AdamW
optimizer [41] with betas 0.8 and 0.99, weight decay 0.01, and an initial
learning rate of 0.0002 decaying exponentially by a gamma of 0.999875 [42].
The models were trained using an NVIDIA A100 SXM4 80GB with a batch size of
100. All models are released in the Coqui TTS
toolkit111111https://github.com/coqui-ai/TTS.

###  5.1 Results and Discussion

We evaluated the synthesized speech using subjective judgments, averaged
across multiple speakers. We randomly sampled 50 segments from the in-domain
test set, as well as out-of-domain corpora to test the models' ability to
generalize to non-Bible contexts. The out-of-domain sentences were obtained
from the NEWS corpus121212http://github.com/masakhane-io/lacuna_pos_ner
(except for Akuapem Twi). Annotators rated the quality of synthesized speech
in terms of naturalness of voice and appropriateness of pronunciation for the
particular language or dialect. Annotators selected from a 5-point Likert
rating for each sample: 1 (bad), 2 (poor), 3 (fair), 4 (good), and 5
(excellent). The mean opinion scores (MOS) are shown in Table 4. We
additionally use mel cepstral distortion (MCD) [43], an automatic edit
distance metric, to assess quality for the in-domain segments where we have
reference speech, with dynamic time warping (DTW) to align the segments. MCD
largely follows MOS: languages with better human judgments (higher MOS)
typically have better (lower) MCD scores, though MCD can be misleading, as in
Lingala.

The MOS judgments seem related to the goodness of alignment evaluations. That
is, the language with the best alignments (ewe) was also rated the best MOS
for speech synthesized from the resulting model. Similarly, the language with
the worst alignments (hau) resulted in the TTS model with the lowest out-of-
domain MOS scores. To improve MOS, it may be necessary to either improve the
alignments or apply more stringent outlier exclusion criteria, which should be
possible, as the training data size remains significantly larger than many
available TTS corpora for these languages or others.

Table 3: Human evaluation of alignment. Shown are percentages of
<audio,transcript> samples with an exact match (EM), added words, missing
words, or both.

|  |  |  |  | Conflict.
---|---|---|---|---|---
Language | EM | Add. | Miss. | Both | Labels
Éwé | 92% | 2% | 4% | 0% | 2%
Hausa | 32% | 68% | 0% | 0% | 0%
Lingala | 74% | 12% | 0% | 0% | 14%
Akuapem Twi | 88% | 0% | 8% | 2% | 2%
Asante Twi | 78% | 2% | 6% | 6% | 8%
Yorùbá | 76% | 24% | 0% | 0% | 0%

Table 4: Evaluation of TTS model outputs using both human judgments (MOS) and
an automatic metric (MCD). In-Domain texts are Bible verses, and Out-of-Domain
is news.

| MOS | MOS | MCD
---|---|---|---
Language | In-Domain | Out-of-domain |
Éwé | 4.34 | 3.87 | 5.8
Hausa | 3.42 | 2.34 | 7.6
Lingala | 3.31 | 2.40 | 5.6
Akuapem Twi | 2.79 | — | 7.5
Asante Twi | 3.07 | 2.44 | 6.8
Yorùbá | 4.06 | 2.93 | 5.8

##  6 Conclusions

The BibleTTS corpus is the first of its kind in many respects. The quality and
volume of the data is extremely rare in open speech corpora – these are
professional, studio quality, 48kHz recordings, with up to 86 hours of verse-
aligned data per language, for 10 languages spoken in sub-Saharan Africa. The
BibleTTS license is research and commercial friendly: CC-BY-SA. We hope that
this corpus will enable advances in speech technology for African languages
and also will unlock new techniques in TTS, which require more, higher-quality
data.

We described our approach to verse and sentence-level alignment of the
original found data with a variety of different resources. We used human
evaluation to assess the quality of the resulting alignments, and validate the
resulting data by training high-quality speech synthesis models with Coqui
TTS.

There are two clear and immediate avenues for future work: (1) verse-level
alignment of the remaining four languages (kik, lug, luo, and nya), and (2)
improvement of the quality of existing alignments. Given the volume of data
per language, it may well be the case that we can be more conservative with
outlier removal, keeping only 20 or 30 hours of the best data, and obtain even
better resulting TTS models. Nevertheless, we have shown that the data can
already be used to produce high-quality TTS models (as with Ewe), on both in
and out of domain text. We plan to update BibleTTS such that we have high-
quality verse-level alignments for all ten languages.

##  7 Acknowledgements

We are very grateful to the volunteers, Richard J. Bonnie, Komlanvi D. Akoly,
Komlanvi M. Klove, Ibrahim Haruna, Oluwabusayo O. Awoyomi, Emmanuel Anebi,
Christian Kilapi, Pacifick Taba, who helped with human evaluation and the
Masakhane community.

## References

  * [1] D. M. Eberhard, G. F. Simons, and C. D. F. (eds.), ``Ethnologue: Languages of the world. twenty-third edition.'' SIL International, 2022. [Online]. Available: http://www.ethnologue.com
  * [2] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, ``Librispeech: an asr corpus based on public domain audio books,'' in _ICASSP_. IEEE, 2015.
  * [3] H. Zen, V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, ``Libritts: A corpus derived from librispeech for text-to-speech,'' _Interspeech_ , 2019.
  * [4] K. Ito and L. Johnson, ``The lj speech dataset,'' https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [5] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan _et al._ , ``Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,'' in _ICASSP_. IEEE, 2018.
  * [6] R. Valle, K. Shih, R. Prenger, and B. Catanzaro, ``Flowtron: an autoregressive flow-based generative network for text-to-speech synthesis,'' _arXiv preprint arXiv:2005.05957_ , 2020.
  * [7] J. Kim, J. Kong, and J. Son, ``Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,'' in _International Conference on Machine Learning_. PMLR, 2021\.
  * [8] A. Purington, J. G. Taft, S. Sannon, N. N. Bazarova, and S. H. Taylor, ``" alexa is my new bff" social roles, user satisfaction, and personification of the amazon echo,'' in _Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems_ , 2017.
  * [9] P. Dempsey, ``The teardown: Google home personal assistant,'' _Engineering & Technology_, 2017.
  * [10] E. Casanova, A. C. Junior, C. Shulby, F. S. d. Oliveira, J. P. Teixeira, M. A. Ponti, and S. Aluísio, ``Tts-portuguese corpus: a corpus for speech synthesis in brazilian portuguese,'' _Language Resources and Evaluation_ , 2022.
  * [11] A. W. Black, ``Cmu wilderness multilingual speech dataset,'' in _ICASSP_. IEEE, 2019.
  * [12] M. Harper, ``The IARPA Babel multilingual speech database,'' 2011. [Online]. Available: https://www.iarpa.gov/index.php/research-programs/babel
  * [13] E. Salesky, M. Wiesner, J. Bremerman, R. Cattoni, M. Negri, M. Turchi, D. W. Oard, and M. Post, ``Multilingual tedx corpus for speech recognition and translation,'' in _Proceedings of Interspeech_ , 2021.
  * [14] D. van Niekerk, E. Barnard, O. Giwa, and A. Sosimi, ``Lagos-nwu yoruba speech corpus,'' 2015.
  * [15] D. R. van Niekerk and E. Barnard, ``Tone realisation in a yorùbá speech recognition corpus,'' in _SLTU_ , 2012.
  * [16] A. Gutkin, I. Demirsahin, O. Kjartansson, C. E. Rivera, and K. Túbòsún, ``Developing an open-source corpus of yoruba speech,'' 2020.
  * [17] A. Öktem, M. A. Jaam, E. DeLuca, and G. Tang, ``Gamayun - language technology for humanitarian response,'' in _2020 IEEE Global Humanitarian Technology Conference (GHTC)_ , 2020.
  * [18] T. K. Dagba, J. O. R. Aoga, and C. C. Fanou, ``Design of a yoruba language speech corpus for the purposes of text-to-speech (tts) synthesis,'' in _ACIIDS_ , 2016.
  * [19] A. A. Afolabi, E. O. Omidiora, and O. T. Arulogun, ``Development of text to speech system for yoruba language,'' 2014.
  * [20] A. E. Akinwonmi and B. K. Alese, ``A prosodic text-to-speech system for yorùbá language,'' _ICITST_ , 2013.
  * [21] O. O. Àjàdí, ``A quantitative model of yorùbá speech intonation using stem-ml,'' _INFOCOMP Journal of Computer Science_ , 2007\.
  * [22] O. À. Odéjobí, A. J. Beaumont, and S. H. S. Wong, ``A computational model of intonation for yorùbá text-to-speech synthesis: Design and analysis,'' in _TSD_ , 2004.
  * [23] D. van Niekerk, C. van Heerden, M. Davel, N. Kleynhans, O. Kjartansson, M. Jansche, and L. Ha, ``Rapid development of tts corpora for four south african languages,'' 2017.
  * [24] E. Barnard, M. H. Davel, C. van Heerden, F. De Wet, and J. Badenhorst, ``The nchlt speech corpus of the south african languages.'' Workshop Spoken Language Technologies for Under-resourced Languages (SLTU), 2014.
  * [25] A. W. Black, P. A. Taylor, and R. Caley, ``Festival speech synthesis system,'' 1998\.
  * [26] M. Schröder, M. Charfuelan, S. Pammi, and I. Steiner, ``Open source voice creation toolkit for the mary tts platform,'' in _INTERSPEECH_ , 2011.
  * [27] A. R. Iyanda and O. D. Ninan, ``Development of a yorúbà textto-speech system using festival,'' _Innovative Systems Design and Engineering (ISDE)_ , vol. 8, no. 5, 2017.
  * [28] J. O. Aoga, T. K. Dagba, and C. C. Fanou, ``Integration of yoruba language into marytts,'' _International Journal of Speech Technology_ , 2016.
  * [29] M. E. Ekpenyong, E.-A. Urua, and D. Gibbon, ``Towards an unrestricted domain tts system for african tone languages,'' _International Journal of Speech Technology_ , 2008.
  * [30] S. H. Mariam, K. Prahallad, A. W. Black, R. Kumar, and R. Sangal, ``Unit selection voice for amharic using festvox,'' in _SSW_ , 2004.
  * [31] T. K. Dagba and C. Y. Boco, ``A text to speech system for fon language using multisyn algorithm,'' in _KES_ , 2014.
  * [32] J. A. Louw, M. H. Davel, and E. Barnard, ``A general-purpose isizulu speech synthesizer,'' _South African Journal of African Languages_ , 2005.
  * [33] M. Gakuru, F. K. Iraki, R. C. F. Tucker, K. B. Shalonova, and K. Ngugi, ``Development of a kiswahili text to speech system,'' in _INTERSPEECH_ , 2005\.
  * [34] M. E. Ekpenyong, E.-A. Urua, O. Watts, S. King, and J. Yamagishi, ``Statistical parametric speech synthesis for ibibio,'' _Speech Communication_ , 2014.
  * [35] A. R. Ìyàndá, O. A. Odejobi, F. A. Soyoye, and O. O. Akinadé, ``Development of grapheme-to-phoneme conversion system for yorùbá text-to-speech synthesis,'' 2014.
  * [36] O. O. Akinadé and d. A. Ọdẹ́jọbí, ``Computational modelling of yorùbá numerals in a number-to-text conversion system,'' _Journal of Language Modelling_ , 2014.
  * [37] M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, ``Montreal Forced Aligner: Trainable text-speech alignment using Kaldi,'' in _Proc. Interspeech_ , Stockholm, Sweden, 2017.
  * [38] T. Schultz, N. T. Vu, and T. Schlippe, ``Globalphone: A multilingual text amp; speech database in 20 languages,'' in _2013 IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2013.
  * [39] J. Kim, S. Kim, J. Kong, and S. Yoon, ``Glow-tts: A generative flow for text-to-speech via monotonic alignment search,'' _Advances in Neural Information Processing Systems_ , 2020.
  * [40] K. Ito _et al._ , ``The lj speech dataset,'' 2017.
  * [41] I. Loshchilov and F. Hutter, ``Decoupled weight decay regularization,'' 2017.
  * [42] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga _et al._ , ``Pytorch: An imperative style, high-performance deep learning library,'' _Advances in neural information processing systems_ , 2019.
  * [43] J. Kominek, T. Schultz, and A. W. Black, ``Synthesizer voice quality of new languages calibrated with mean mel cepstral distortion,'' in _Spoken Languages Technologies for Under-Resourced Languages_ , 2008.


# Energy relaxation properties of a microwave resonator coupled to a pumped
two-level system bath

Thibault Capelle  Laboratoire Kastler Brossel, Sorbonne Université, CNRS, ENS-
Université PSL, Collège de France, 75005 Paris, France  Emmanuel Flurin
Département de Physique, ENS-Université PSL, CNRS, 24 rue Lhomond, F-75005
Paris, France  Edouard Ivanov  Laboratoire Kastler Brossel, Sorbonne
Université, CNRS, ENS-Université PSL, Collège de France, 75005 Paris, France
Jose Palomo  Michael Rosticher  Laboratoire Pierre Aigrain,ENS-Université PSL,
Sorbonne Université, CNRS, 24 rue Lhomond, 75231 Paris, France  Sheon Chua
Tristan Briant  Pierre-François Cohadon  Antoine Heidmann  Thibaut Jacqmin
Samuel Deléglise  Laboratoire Kastler Brossel, Sorbonne Université, CNRS, ENS-
Université PSL, Collège de France, 75005 Paris, France

###### Abstract

While the main loss channel of planar microwave superconducting resonators has
been identified to be related to an external coupling to a two-level system
bath, the behavior of such a cavity in an environment strongly pumped by a
detuned tone has yet to be fully understood. Here, a semi-classical
theoretical model describing the interaction of such resonators with the Two-
Level Systems is derived, to which experimental datas obtained with different
substrate materials are fitted. Performing the experiments with different
substrates allows the tailoring of fundamental properties of the interacting
two-level system bath.

††preprint: APS/123-QED

Microwave superconducting resonators are an ubiquitous resource in various
quantum devices, ranging from kinetic inductance bolometers Sauvageau and
Macdonald (1989); Audley, Kelley, and Rawley (1993), parametric amplifiers Roy
and Devoret (2016); Pillet  _et al._ (2015), and embedded circuits for cavity
quantum electrodynamics Blais  _et al._ (2004); Rau, Johansson, and Shnirman
(2004); Wallraff  _et al._ (2004). In addition, they play a central role as a
microwave interface for various hybrid quantum systems such as solid-state
spins André  _et al._ (2006), mechanical resonators Teufel  _et al._ (2011);
Tóth  _et al._ (2017), and ferromagnetic magnons Tabuchi  _et al._ (2014,
2015). Owing to the very low resistivity of superconducting materials,
combined with advanced electromagnetic engineering of radiation losses, the
dominant loss channel of such resonators is the dielectric losses due to the
presence of a two-level system (TLS) bath in amorphous materials Wang  _et
al._ (2009); Barends  _et al._ (2008). Due to the non-linear nature of the
TLS, this loss channel is found to be dependent on the resonator occupancy,
originating from thermal fluctuations Gao  _et al._ (2008), from resonant
excitation Lindström  _et al._ (2009); Skacel  _et al._ (2015) or from non-
degenerate resonant mode occupancy Sage  _et al._ (2011); Kirsh  _et al._
(2017). Moreover, an increasing effort has been devoted to the
characterization of the microscopic properties of individual TLS probed under
stress Lisenfeld  _et al._ (2015) or DC voltage bias Sarabi  _et al._ (2016).
However, in many cases, strong microwave tones are applied with significant
frequency detuning with respect to the resonance frequency Regal, Teufel, and
Lehnert (2008); Rocheleau  _et al._ (2010) in order to activate a parametric
interaction between the resonator mode and another degree of freedom. In this
configuration, the off-resonant microwave tone responsible for the saturation
of the TLS bath is spectrally distinct from the resonant mode subjected to the
losses, giving simultaneous rise to a shift of the resonance frequency and a
modification of the population dependency of the quality factor.

In this letter, we derive a semi-classical model that accounts for the energy
relaxation properties of a microwave resonator due to the presence of an off-
resonance pumped TLS bath. This model reproduces the experimental
characterizations conducted on lumped-element coplanar resonators fabricated
on various substrates. From these measurements, the model allows the
extraction of the average spectral distribution of the TLS, as well as the
relaxation properties and typical coupling rate to the resonator. This study
covers a wide range of experimental situations as of yet unexplored. Due to
the rapid rise of hybrid quantum circuits, it is anticipated that the model
and methodology will be widely applicable in the design and characterization
of superconducting resonators, and provides a new tool for the investigation
of the spectral structure of amorphous materials.

Figure 1:  a) Optical micrograph of the microwave cavity. b) The dielectric
environment of the capacitance (left) and the profile of the electric field
generated (right). Silicon is represented in purple, niobium is represented in
grey, and the optional amorphous dielectric layer is represented in green. c)
zoom on the capacitor using electron microscopy.

In this work, we study lumped-element resonators, in which an interdigitated
capacitor and a meander inductor are connected in parallel. This geometry was
optimized to maximize the coupling of the microwave mode to an external
degree-of-freedom such as localized emitters Douce  _et al._ (2015); Haikka
_et al._ (2017) or the motion of a planar dielectric membrane placed in the
evanescent field of the electrodes via dielectric gradient forces
Unterreithmeier, Weig, and Kotthaus (2009). A good approximation of the
electrostatic potential in the vicinity of the interdigitated capacitor is
given by the harmonic approximation Den Otter (2002)

$V(x,z)=V_{0}\sin(2\pi x/a)e^{-2\pi|z|/a},$ (1)

where $a$ is the period of the electrodes oriented along the $y$-axis and
interleaved along the $x$-axis. This shows that the nominal extent of the
electrostatic potential is inversely proportional to the period, and that the
electrostatic energy can thus be confined in an arbitrarily small region
around the electrode plane $z=0$ by reducing $a$. This confinement is
advantageous in the context of hybrid quantum circuits, where coupling of an
electromagnetic mode to a well-controlled additional system is desirable.
However, this makes the resonator highly sensitive to surface defects located
at the various interfaces between superconducting material, substrate, and
vacuum. The interaction with these defects, described by the coupling to a TLS
bath, is the dominant source of dissipation in planar resonators Geerlings
_et al._ (2012) and constitutes the main subject of this article. We have
fabricated such lumped-element resonators with a small interdigitated
electrode period $a=1\,\mu$m to confine the field in a layer of thickness
$\sim 200$ nm around the electrode plane.

Fig. 1 shows an optical micrograph of such a resonator. The meander, of length
$l\sim 5$ mm, creates an inductance $L\approx\mu_{0}l$ in the 5 nH range. In
addition, as a consequence of Eq. (1), the capacitance of the interdigitated
capacitor is given by (see supplementary material)

$C=S\epsilon_{0}(1+\epsilon_{\mathrm{s}})/a,$ (2)

where $S$ is the area of the interdigitated electrodes and
$\epsilon_{\mathrm{s}}$ is the relative permittivity of the substrate below
the electrodes. We have chosen $S$ such that $C\sim 0.12$ fF, resulting in a
resonance frequency $\omega_{\mathrm{c}}/2\pi=1/2\pi\sqrt{LC}\approx 6$ GHz.
Finally, the resonator is probed by a feedline with an inductive coupling rate
$\Gamma_{\mathrm{ext}}/2\pi\approx 0.2$ MHz.

Figure 2: a) Experimental setup. b) Experimental protocol: a strong pump field
is applied far-detuned from a microwave cavity resonance, while a weak probe
field is swept across the cavity resonance to measure its transmission
spectrum $\mathcal{T}$. c) Real (top) and imaginary part (bottom) of the
transmission for various intracavity photon numbers.

To probe the influence of the TLS bath on the resonator, we have performed a
pump-probe experiment: a strong pump, detuned by several linewidths from the
cavity resonance, saturates the TLS bath and a weak probe is used to measure
the resonance frequency $\omega_{c}/2\pi$ and full damping $\Gamma/2\pi$ of
the resonator (See Figure 2). These quantities are extracted from the probe
transmission spectrum $\mathcal{T}$ (see Figure 2c) using a fit formula that
properly takes into account the effect of standing waves in the coupling
waveguide Geerlings  _et al._ (2012). Notably, this procedure allows to
precisely extract the internal losses $\Gamma_{i}$ from the total cavity
linewidth $\Gamma=\Gamma_{\mathrm{i}}+\Gamma_{\mathrm{ext}}$. While the weak
probe has a negligible effect on the TLS bath, the pump affects the population
imbalance $\langle\sigma_{z}(\omega_{\mathrm{q}})\rangle$ according to the
saturation law for a TLS at frequency $\omega_{\mathrm{q}}$

$\langle\sigma_{z}\rangle=\langle\sigma_{z}\rangle_{\mathrm{th}}\left(1-\frac{\Gamma_{2}^{2}\mathcal{C}}{(\omega_{\mathrm{q}}-\omega_{\mathrm{p}})^{2}+\Gamma_{2}^{2}(1+\mathcal{C})}\right),$
(3)

where
$\langle\sigma_{z}\rangle_{\mathrm{th}}=-\tanh\left(\hbar\omega_{\mathrm{q}}/2k_{B}T\right)\approx-0.52$
is the thermal imbalance resulting from the Fermi-Dirac distribution at the
base temperature $T=330$ mK of our 3He cryostat, $\Gamma_{2}$ ($\Gamma_{1}$)
the TLS dephasing (energy relaxation) rate,
$\mathcal{C}=4ng^{2}/\Gamma_{1}\Gamma_{2}$ the cooperativity of the TLS given
the coupling rate $g$, and the number of pump intracavity photons $n$ at the
angular frequency $\omega_{\mathrm{p}}$. In turn, the population difference
$\langle\sigma_{z}(\omega_{\mathrm{q}})\rangle$ of a single TLS induces a
shift of the complex cavity frequency Kirsh  _et al._ (2017)

$\delta\omega=\frac{g^{2}\langle\sigma_{z}(\omega_{\mathrm{q}})\rangle}{\omega_{\mathrm{q}}-\omega_{\mathrm{c}}+i\Gamma_{2}}.$
(4)

The frequency shift and damping, as measured by the probe beam, are related to
the real and imaginary parts of $\delta\omega$. The total frequency shift and
damping are obtained by integrating the contribution of individual TLS,
assuming a flat spectral distribution of density $P_{0}$, uniform coupling
rate $g$, and no interaction between individual TLSs, leading to (see
supplementary material)

$\Delta\omega_{\mathrm{c}}=-\frac{P_{\mathrm{0}}g^{2}\langle\sigma_{z}\rangle_{\mathrm{th}}}{2}\frac{\mathcal{C}}{\sqrt{1+\mathcal{C}}}\frac{\delta}{\delta^{2}+(1+\sqrt{1+\mathcal{C}})^{2}},$
(5)
$\Gamma_{c}=-P_{\mathrm{0}}g^{2}\langle\sigma_{z}\rangle_{\mathrm{th}}\left[1-\frac{\mathcal{C}}{\sqrt{1+\mathcal{C}}}\frac{1+\sqrt{1+\mathcal{C}}}{\delta^{2}+(1+\sqrt{1+\mathcal{C}})^{2}}\right].$
(6)

In these formulas,
$\delta=(\omega_{\mathrm{p}}-\omega_{\mathrm{c}})/\Gamma_{2}$ is the
normalized pump detuning with respect to the resonator frequency. In
particular, the limiting case $\delta=0$ describes the result of single-tone
experiments where the probe tone itself is responsible for the TLS-bath
saturation. In this case, $\Delta\omega_{\mathrm{c}}=0$ and
$\Gamma_{\mathrm{c}}$ monotonically decreases towards $0$ as $\mathcal{C}$
tends to infinity. In contrast, the detuned pump experiment gives rise to a
rich set of predictions: while the monotonous decrease of $\Gamma_{c}$ is
barely modified, a non-zero frequency shift $\Delta\omega_{\mathrm{c}}$ is
predicted for intermediate values of $\mathcal{C}$. The physical
interpretation of this phenomenon is illustrated in Figure 3c (3d) showing the
ground state population as a function of TLS frequency for a positive
(negative) pump detuning $\Delta=\omega_{\mathrm{p}}-\omega_{\mathrm{c}}$ and
various pump powers. At low pump power $\mathcal{C}\ll 1$, the TLS transition
is not saturated, such that only the TLS resonant with the pump
($|\omega_{\mathrm{q}}-\omega_{\mathrm{p}}|<\Gamma_{2}$) are excited. The
corresponding depletion in ground-state population, increasing with
$\mathcal{C}$, gives rise to a frequency pull of the cavity frequency towards
the pump frequency, as predicted by Eq. 4. On the other hand, in the limit of
a saturated transition $\mathcal{C}\gg 1$, the width of the depletion in
ground state population $\Gamma_{2}\sqrt{1+\mathcal{C}}$ can exceed the pump-
cavity detuning $\Delta$, such that the asymmetry of the ground-state
population with respect to $\omega_{\mathrm{c}}$ vanishes, hence,
$\Delta\omega_{\mathrm{c}}$ also tends to 0 for $\mathcal{C}\gg 1$.

The combined dependence of $\Delta\omega_{\mathrm{c}}$ and
$\Gamma_{\mathrm{c}}$ as a function of pump detuning and pump power can be
used to separately extract the TLS spectral density $P_{0}$, their coupling to
the resonator mode characterized by the coupling rate $g$, and their
relaxation rate $\Gamma_{2}$. In the following, we assume
$\Gamma_{1}=2\Gamma_{2}$ as the dependence over $\Gamma_{1}$ cannot be
resolved independently using Eqs. (5) and (6).

Figure 3:  (a) Shift of the cavity frequency and (b) damping of the resonator,
versus the number of intracavity photons. Each color corresponds to a
different detuning. Points correspond to experimental data, while solid lines
correspond to fits from the model. (c) and (d) are computed ground state
population distribution of the two-level systems versus the frequency detuning
$\omega_{\mathrm{q}}-\omega_{\mathrm{c}}$, respectively for the red-circled
and pink-circled points in (a) and (b). This measurement was performed on the
sample Si/SiO2, with $\omega_{c}/2\pi=$ 7.521 GHz (see table 1).

To validate the method, measurements have been performed on different
resonators, fabricated on 3 different samples that are expected to yield very
distinct TLS populations near the substrate interface:

  * •

(Si) A 250 $\mu$m substrate of FZ grown (100) intrinsic silicon, with a
resistivity of more than 10 000 $\Omega$.cm,

  * •

(Si/SiO2) 500 $\mu$m of FZ grown (100) P-doped type-n silicon, with a
resistivity of more than 10 000 $\Omega$.cm, with 2 $\mu$m of SiO2 from
thermal oxidation,

  * •

(Si/SiN) 650 $\mu$m of CZ grown (100) silicon, with P/Boron doping and a
resistivity of 1-30 $\Omega$ cm, with 200 nm of Si3N4 deposited through LPCVD.

In every case, 10 nearly identical resonators are realized by electron-beam
lithography on a 100 nm thick niobium layer evaporated using a Plassys
evaporator under ultra-high vacuum ($\approx 5\times 10^{-10}$ mbar). The
resonators have been multiplexed on the same coupling waveguide and their
resonance frequency was staggered by incrementing the capacitor’s area by
steps of $\sim 2.5$ %. 6 resonances out of 10 have been observed for the Si
substrate, 4 for the Si/Si3N4 substrate, and 3 for the Si/SiO2 substrate. The
missing resonances are attributed to short circuits between the interdigitated
electrodes of the resonator.

To experimentally extract the model parameters, the frequency shift
$\Delta\omega_{\mathrm{c}}$ and intrinsic damping $\Gamma_{\mathrm{i}}$ are
measured systematically for various intracavity pump photon numbers $n_{j}$
and detunings $\Delta_{j}$, for $j\in\\{0,N_{\mathrm{meas}}\\}$ using the
setup in Figure 2. An example of such a measurement is given in Figure 3,
together with a fit using Eqs. 5 and 6. In addition to $P_{0}$, $g$, and
$\Gamma_{2}$, the only free parameters are $\Gamma_{\infty}$, an additive
source of loss unrelated to the TLS bath
($\Gamma_{\mathrm{i}}=\Gamma_{\mathrm{c}}+\Gamma_{\infty}$), and the natural
frequency of the resonator $\omega_{\mathrm{c}}$. Despite the small number of
fit parameters the agreement between theory and experiment is excellent, which
justifies a posteriori the assumptions of our model. The error bars on the
various parameters are calculated by a bootstrap method: a random subsample of
the data points
$\left\\{\Delta\omega_{\mathrm{c}}(n_{i},\Delta_{i}),\Gamma_{\mathrm{c}}(n_{i},\Delta_{i})\right\\}_{i\in[0..N_{\text{meas}}]}$
is chosen for parameter estimation (the number of points in the subsample is
equal to the number of measurements $N_{meas}$, however multiple identical
datapoints are not excluded). The operation is repeated 1000 times to
determine measurement mean value and standard deviation.

The extracted parameters for the three types of resonators are listed in Table
1, together with a 95 % confidence interval. As foreseen, the samples that are
realized on a bare silicon substrate, which should contain very few TLS
defects, since no macroscopic oxidation has been performed, have the smallest
TLS damping. This is characterized by the value $\Gamma^{0}_{c}=P_{0}g^{2}$,
which corresponds to the total TLS damping extrapolated at 0-temperature, and
0-pump power. We find an average value of
$\langle\Gamma^{0}_{c}\rangle=2\pi\times$0.33 MHz for Si resonators,
$2\pi\times 1.2$ MHz for Si/Si3N4 resonators and $2\pi\times 1.5$ MHz for
Si/SiO2 respectively. As apparent on the confidence intervals, this parameter
is very well estimated by the model. Indeed, it directly relates to the
measured damping at low pump power. The individual extraction of $P_{0}$ and
$g$ is made possible by fitting the shape of $\Delta\omega_{\mathrm{c}}$ and
$\Gamma_{\mathrm{i}}$, albeit with a smaller confidence interval. For Si/SiO2
samples, the larger damping is clearly explained by a larger TLS spectral
density $\langle P_{0}\rangle\sim$4.4 times larger than for Si samples. On the
other hand, surpisingly, Si/Si3N4 resonators have a TLS spectral density
comparable to that of Si resonators, but individual TLS couplings $\langle
g\rangle\sim$2.2 times larger, probably originating from the strong overlap
between the electric field profile and the 200 nm amorphous layer. The
variability of the extracted parameters between similar resonators can be
explained by statistical fluctuations in the actual TLSs distribution. Indeed,
the number of TLSs effectively interacting with the resonator is as low as
$n_{\mathrm{TLS}}=P_{0}\Gamma_{2}/2\pi\sim 4$ on average. Finally, let’s note
that the average coupling $g$ determined in this work is approximately a
factor 4 larger than previously reported in planar resonators fabricated on Si
substrates Kirsh  _et al._ (2017). This is likely a consequence of the sub-
micron confinement of the electric field in our interdigitated capacitors.

Type of wafer |  $\Gamma_{\mathrm{c}}^{0}/2\pi$ (kHz) |  $g/2\pi$ (kHz) |  $P_{0}$ (MHz-1) |  $\Gamma_{1}/2\pi$ (kHz) |  $\omega_{\mathrm{c}}/2\pi$ (GHz)
---|---|---|---|---|---
Si/Si3N4 | $2000\pm 73$ | $540\pm 98$ | $1.1\pm 0.4$ | $3400\pm 1000$ | $6.037$
Si/Si3N4 | $1200\pm 86$ | $510\pm 90$ | $0.79\pm 0.31$ | $3500\pm 1400$ | $5.958$
Si/Si3N4 | $620\pm 69$ | $440\pm 170$ | $0.56\pm 0.4$ | $4300\pm 2200$ | $5.886$
Si/Si3N4 | $790\pm 90$ | $240\pm 59$ | $2.3\pm 1.4$ | $5300\pm 2500$ | $5.742$
Si | $300\pm 22$ | $200\pm 60$ | $1.3\pm 0.39$ | $2200\pm 820$ | $5.069$
Si | $300\pm 22$ | $170\pm 43$ | $1.8\pm 0.52$ | $2200\pm 710$ | $5.300$
Si | $330\pm 19$ | $200\pm 44$ | $1.3\pm 0.36$ | $2400\pm 790$ | $5.378$
Si | $330\pm 22$ | $150\pm 27$ | $2.3\pm 0.54$ | $2500\pm 710$ | $5.229$
Si | $280\pm 23$ | $160\pm 44$ | $1.8\pm 0.57$ | $2300\pm 800$ | $5.114$
Si | $410\pm 24$ | $220\pm 37$ | $1.4\pm 0.41$ | $1000\pm 840$ | $5.443$
Si/SiO2 | $1400\pm 28$ | $380\pm 33$ | $1.5\pm 0.25$ | $2700\pm 870$ | $7.657$
Si/SiO2 | $1900\pm 50$ | $170\pm 10$ | $11\pm 1.4$ | $2800\pm 440$ | $7.521$
Si/SiO2 | $1300\pm 18$ | $150\pm 11$ | $10\pm 1.4$ | $1300\pm 440$ | $7.109$
Table 1: Extracted parameters of the TLS bath for the three types of substrate

In conclusion, we have presented an experimental method to characterize the
non-linear properties of TLS bath. The method has been applied to lumped-
element resonators that have been specifically optimized to confine the
electric field in a small region around the substrate surface. By selectively
saturating a fraction of the TLSs that is resonant with a strong detuned-pump
field, and simultaneously measuring the cavity spectrum with a weak probe
field, we have observed clear signatures of the interaction with the non-
linear bath. The details of the evolution of the resonance frequency and
damping as a function of the pump detuning can be used to infer physical
properties of the bath, such as the spectral density of TLS, their coupling to
the resonator, and their lifetime. The prerequisite of this technique boils
down to standard microwave equipment and constitutes a useful tool to
precisely characterize the properties of TLS bath - the dominant loss channel
of planar superconducting resonators. Furthermore, the physical situation
considered here is ubiquitous in parametrically coupled systems where a
particular interaction is activated by a strong pump-field detuned with
respect to the resonant mode frequency. In this regime, the coupled dynamic of
the resonant mode and TLS bath has to be carefully studied since pump-photons
can be scattered to the resonator via the interaction with TLSs, leading to an
effective heating process. This phenomenon is currently under theoretical
investigation Forni  _et al._ (2018) and its characterization, which requires
a quantum limited read-out to resolve the associated fluctuations, will be the
subject of future work.

###### Acknowledgements.

The authors would like to thank Paolo Forni and Alain Sarlette for insightful
discussions, Takis Kontos for his help with the Niobium deposition facility,
Sundae Chen, Leo Morel and Leonhard Neuhaus for early developments. This work
was supported by a starting grant from ”Agence Nationale pour la Recherche”
QuNaT (ANR-14-CE26-0002). E.F. is funded by a Junior research chair from the
LabEX ENS-ICFP: ANR-10-LABX-0010/ ANR-10-IDEX-0001-02 PSL*. E.I. acknowledges
support from the European Union’s Horizon 2020 Programme for Research and
Innovation under grant agreement No. 722923 (Marie Curie ETN - OMT). S. C. is
supported by the grant ANR ExSqueez (ANR-15-CE30-0014).

## References

  * Sauvageau and Macdonald (1989) J. Sauvageau and D. Macdonald, IEEE Transactions on magnetics 25, 1331 (1989).
  * Audley, Kelley, and Rawley (1993) M. D. Audley, R. L. Kelley, and G. L. Rawley, Journal of Low temperature physics 93, 245 (1993).
  * Roy and Devoret (2016) A. Roy and M. Devoret, Comptes Rendus Physique 17, 740 (2016).
  * Pillet  _et al._ (2015) J.-D. Pillet, E. Flurin, F. Mallet, and B. Huard, Applied Physics Letters 106, 222603 (2015).
  * Blais  _et al._ (2004) A. Blais, R.-S. Huang, A. Wallraff, S. M. Girvin, and R. J. Schoelkopf, Physical Review A 69, 062320 (2004).
  * Rau, Johansson, and Shnirman (2004) I. Rau, G. Johansson, and A. Shnirman, Physical Review B 70, 054521 (2004).
  * Wallraff  _et al._ (2004) A. Wallraff, D. Schuster, A. Blais, L. Frunzio, R.-S. Huang, J. Majer, S. Kumar, S. Girvin, and R. Schoelkopf, Nature 461, 162 (2004).
  * André  _et al._ (2006) A. André, D. DeMille, J. M. Doyle, M. D. Lukin, S. E. Maxwell, P. Rabl, R. J. Schoelkopf, and P. Zoller, Nature Physics 2, 636 (2006).
  * Teufel  _et al._ (2011) J. Teufel, T. Donner, L. Dale, J. Harlow, M. S. Allman, K. Cicak, A. J. Sirois, J. D. Whittaker, K. W. Lehnert, and R. W. Simmonds, Nature 475, 359 (2011).
  * Tóth  _et al._ (2017) L. Tóth, N. R. Bernier, A. Nunnenkamp, A. K. Feofanov, and T. J. Kippenberg, Nature 13, 783 (2017).
  * Tabuchi  _et al._ (2014) Y. Tabuchi, S. Ishino, T. Ishikawa, R. Yamazaki, K. Usami, and Y. Nakamura, Physycal Review Letter 113, 083603 (2014).
  * Tabuchi  _et al._ (2015) Y. Tabuchi, S. Ishino, A. Noguchi, T. Ishikawa, R. Yamazaki, K. Usami, and Y. Nakamura, Science (2015), 10.1126/science.aaa3693.
  * Wang  _et al._ (2009) H. Wang, M. Hofheinz, J. Wenner, M. Ansmann, R. C. Bialczak, M. Lenander, E. Lucero, M. Neeley, A. D. O’Connell, D. Sank, M. Weides, A. N. Cleland, and J. M. Martinis, Applied Physics Letters 95, 233508 (2009).
  * Barends  _et al._ (2008) R. Barends, H. L. Hortensius, T. Zijlstra, J. J. A. Baselmans, S. J. C. Yates, J. R. Gao, and T. M. Klapwijk, Applied Physics Letters 92, 223502 (2008).
  * Gao  _et al._ (2008) J. Gao, M. Daal, J. M. Martinis, A. Vayonakis, J. Zmuidzinas, B. Sadoulet, B. A. Mazin, P. K. Day, and H. G. Leduc, Applied Physics Letters 92, 212504 (2008).
  * Lindström  _et al._ (2009) T. Lindström, J. E. Healey, M. S. Colclough, C. M. Muirhead, and A. Y. Tzalenchuk, Physical Review B 80, 132501 (2009).
  * Skacel  _et al._ (2015) S. T. Skacel, C. Kaiser, S. Wuensch, H. Rotzinger, A. Lukashenko, M. Jerger, G. Weiss, M. Siegel, and A. V. Ustinov, Applied Physics Letters 106, 022603 (2015).
  * Sage  _et al._ (2011) J. M. Sage, V. Bolkhovsky, W. D. Oliver, B. Turek, and P. B. Welander, Journal of Applied Physics 109, 063915 (2011).
  * Kirsh  _et al._ (2017) N. Kirsh, E. Svetitsky, A. L. Burin, M. Schechter, and N. Katz, Physical Review Materials 1, 012601 (2017).
  * Lisenfeld  _et al._ (2015) J. Lisenfeld, G. J. Grabovskij, C. Müller, J. H. Cole, G. Weiss, and A. V. Ustinov, Nature communications 6, 6182 (2015).
  * Sarabi  _et al._ (2016) B. Sarabi, A. N. Ramanayaka, A. L. Burin, F. C. Wellstood, and K. D. Osborn, Physical Review Retters 116, 167002 (2016).
  * Regal, Teufel, and Lehnert (2008) C. Regal, J. Teufel, and K. Lehnert, Nature Physics 4, 555 (2008).
  * Rocheleau  _et al._ (2010) T. Rocheleau, T. Ndukum, C. Macklin, J. B. Hertzberg, A. A. Clerck, and K. C. Schwab, Nature 463, 72 (2010).
  * Douce  _et al._ (2015) T. Douce, M. Stern, N. Zagury, P. Bertet, and P. Milman, Physical Review A 92, 052335 (2015).
  * Haikka  _et al._ (2017) P. Haikka, Y. Kubo, A. Bienfait, P. Bertet, and K. Mølmer, Physycal Review A 95, 022306 (2017).
  * Unterreithmeier, Weig, and Kotthaus (2009) Q. P. Unterreithmeier, E. M. Weig, and J. P. Kotthaus, Nature 458, 1001 (2009).
  * Den Otter (2002) M. Den Otter, Sensors and Actuators, A 96, 140 (2002).
  * Geerlings  _et al._ (2012) K. Geerlings, S. Shankar, E. Edwards, L. Frunzio, R. J. Schoelkopf, and M. H. Devoret, Applied Physics Letters 100, 192601 (2012).
  * Forni  _et al._ (2018) P. Forni, A. Sarlette, T. Capelle, E. Flurin, S. Deléglise, and P. Rouchon, (2018), arXiv:1803.07810 [quant-ph] .


# Measuring the distortion of time with relativistic effects in large-scale
structure

Daniel Sobral Blanco, Camille Bonvin
Départment de Physique Théorique and Center for Astroparticle Physics,
Université de Genève, Quai E. Ansermet 24, CH-1211 Genève 4, Switzerland
daniel.sobralblanco<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

To test the theory of gravity one needs to test, on one hand, how space and
time are distorted by matter and, on the other hand, how matter moves in a
distorted space-time. Current observations provide tight constraints on the
motion of matter, through the so-called redshift-space distortions, but they
only provide a measurement of the sum of the spatial and temporal distortions,
via gravitational lensing. In this Letter, we develop a method to measure the
time distortion on its own. We show that the coming generation of galaxy
surveys, like the Square Kilometer Array, will allow us to measure the
distortion of time with an accuracy of 10-30%. Such a measurement will be
essential to test deviations from the $\Lambda$CDM model in a fully model-
independent way. In particular, it can be used to compare the spatial and
temporal distortions of space-time and to unambiguously distinguish between
modifications of gravity and dark fifth forces acting on dark matter.

###### keywords:

cosmology: theory – gravitation – large-scale structure of Universe

††pubyear: 2022††pagerange: Measuring the distortion of time with relativistic
effects in large-scale structure–Measuring the distortion of time with
relativistic effects in large-scale structure

##  1 Introduction

Two fundamental puzzles are at the core of current research in cosmology: what
is causing the observed accelerated expansion of the Universe; and what is the
nature of dark matter. The standard model of cosmology, the $\Lambda$CDM
model, postulates that the accelerated expansion is due to a cosmological
constant, and that dark matter is a perfect non-relativistic fluid which
interacts only gravitationally with the normal baryonic matter. One of the
main goal of current and future large-scale structure surveys is to test the
validity of the $\Lambda$CDM model, and explore deviations from this minimal
scenario.

Deviations from $\Lambda$CDM are often studied separately by different
communities: some studies concentrate on models beyond General Relativity (see
e.g. Koyama (2016); Joyce et al. (2016) for a review), assuming that dark
matter obey the weak equivalence principle; while other studies explore non-
standard dark matter models subjected to a dark fifth force (see e.g. Frieman
& Gradwohl (1991); Gradwohl & Frieman (1992); Barros et al. (2019); Asghari et
al. (2019); Archidiacono et al. (2022)), assuming the validity of General
Relativity (GR). In practice however, since we do not know if GR is valid and
we do not know if dark fifth forces exist, we would like to use observations
to distinguish between these different scenarios. More precisely, if we
observe deviations from the $\Lambda$CDM model, it is legitimate to ask
ourselves if they are due to a modification of the theory of gravity, or to a
fifth force acting on dark matter 111Note that there are also models where
gravity is modified and in addition the equivalence principle is broken since
dark matter and baryons are coupled differently to the new degree of freedom
(Gleyzes et al., 2015, 2016). In other models the equivalence principle is
valid at the microscopic level, but the modifications of gravity generate an
effective breaking of the equivalence principle due to screening (Hui et al.,
2009)..

One key observable in large-scale structure surveys is the growth rate of
perturbations $f$, that can be measured in a model-independent way from
redshift-space distortions (RSD), see e.g. Alam et al. (2021b). While this
growth of structure provides a powerful way of unveiling deviations from
$\Lambda$CDM, it cannot unambiguously distinguish between modifications of
gravity and dark fifth forces since they impact the way matter is clustering
in a very similar way (Song et al., 2011; Castello et al., 2022; Bonvin &
Pogosian, 2022). A key difference between modifications of gravity and dark
fifth force models however, is the fact that in modified gravity theories,
very generically the two gravitational potentials encoding spatial ($\Phi$)
and temporal ($\Psi$) distortions are different (see e.g. Saltas et al.
(2014)), whereas in GR these potentials are the same at late time. This
property remains of course valid in the presence of a dark fifth force acting
on dark matter (only very exotic models of dark matter with a non-zero
anisotropic stress would modify this.) Hence measuring the relation between
$\Phi$ and $\Psi$, the so-called anisotropic stress, is often refereed to as a
smoking gun for modified gravity, i.e. a way of distinguishing a dark fifth
force from a deviation from GR (Amendola et al., 2013; Song et al., 2010;
Motta et al., 2013; Amendola et al., 2018).

As discussed in Bonvin & Pogosian (2022), this test can however not been
performed with current methods. The reason for this is that, besides RSD, we
are currently only able to measure the sum of the two gravitational
potentials, $\Phi+\Psi$, through gravitational lensing (cosmic shear or CMB
lensing) (Köhlinger et al., 2017; Abbott et al., 2022). We have no individual
measurements of $\Phi$ or $\Psi$. The only way to measure the anisotropic
stress is therefore to use measurements of the galaxy velocity $V$ from RSD,
and translate these into measurements of $\Psi$ assuming the validity of Euler
equation (Motta et al., 2013). While this procedure is perfectly valid in
models of modified gravity, it clearly does not work in models with a dark
fifth force, that break the equivalence principle. Wrongly assuming Euler
equation in this case would lead us to measure an effective difference between
$\Phi$ and $\Psi$, and to erroneously conclude that gravity is modified even
if it is not.

In this Letter, we design a method to solve this problem by measuring $\Psi$
directly from large-scale structure observations. We use the fact that
gravitational redshift leaves an impact on the distribution of galaxies (Yoo
et al., 2009; Bonvin & Durrer, 2011; Challinor & Lewis, 2011), which can be
measured by looking for a dipole in the cross-correlation of bright and faint
galaxies (Yoo et al., 2012; Croft, 2013; Bonvin et al., 2014; Bonvin, 2014;
Gaztanaga et al., 2017; Bonvin et al., 2016; Beutler & Di Dio, 2020; Saga et
al., 2022). Gravitational redshift is an effect predicted by Einstein (1916),
which postulates that time passes at a slower rate in a gravitational
potential well. This effect is directly sensitive to the temporal
gravitational potential $\Psi$. It has been measured in a variety of systems:
around earth, by comparing clocks at different distances from the ground, i.e.
at different values of $\Psi$ (Pound & Snider, 1965; Vessot et al., 1980); in
white dwarf, by looking at the reddening of light due to its escape from the
gravitational field of the star (Adams, 1925; Greenstein & Trimble, 1967); and
in clusters of galaxies and non-linear structures by looking at the difference
in redshift between galaxies at the centre of the structure and galaxies away
from the centre (Wojtak et al., 2011; Sadeh et al., 2015; Alam et al., 2017).
So far, all these measurements have confirmed the validity of GR. However,
this effect (and consequently the potential $\Psi$) has never been measured at
cosmological scales. The goal of this paper is to forecast the precision with
which $\Psi$ will be measured at cosmological scales with the coming
generation of surveys.

We build on a method presented in Sobral-Blanco & Bonvin (2021), where we
showed that by combining different multipoles of the galaxy power spectrum,
one can isolate the correlation of $\Psi$ with the density. For our forecasts
we work with the correlation function, instead of the power spectrum, since
the correlation function can account for wide-angle effects, that contaminate
the measurement of $\Psi$ and need to be properly removed in order not to bias
the measurement. We show that a survey like SKA will be able to measure the
evolution of $\Psi$ with a precision of 10–30 percent.

##  2 Evolution of the gravitational potential

We start by parameterizing the evolution of the gravitational potential in a
model-independent way. We assume that at early time, well in the matter era,
the Universe is well described by GR and Cold Dark Matter (CDM). This has been
tested with great precision by CMB measurements. We link therefore deviations
from the CDM model to the onset of the accelerated expansion of the Universe.
In this context, the matter density perturbations in Fourier space are usually
modelled as

$\displaystyle\delta(\mathbf{k},z)=T_{\delta}(k,z)\Psi_{\rm
in}(\mathbf{k})\,,$ (1)

where $\Psi_{\rm in}$ is the primordial gravitational potential generated by
inflation and $T_{\delta}(k,z)$ is the density transfer function, which
depends on the Fourier mode $k$ and on the redshift $z$. Under the assumption
that deviations from GR appear only at late time, we can write this transfer
function as

$\displaystyle
T_{\delta}(k,z)=\frac{D_{1}(k,z)}{D_{1}(z_{*})}T_{\delta}(k,z_{*})\,,$ (2)

where $T_{\delta}(k,z_{*})$ is the transfer function predicted by GR at
redshift $z_{*}$, well in the matter era. The growth function $D_{1}(k,z)$
describes the growth of structure at late time and depends consequently on the
theory of gravity and on the model of dark energy. At $z_{*}$, where GR is
recovered, this function is independent on $k$.

The velocity transfer function defined through
$V(\mathbf{k},z)=T_{V}(k,z)\Psi_{\rm in}(\mathbf{k})$ is usually related to
the density transfer function using the continuity equation for matter:

$\displaystyle T_{V}(k,z)=-\frac{\mathcal{H}}{k}f(k,z)T_{\delta}(k,z)\,,$ (3)

where $\mathcal{H}$ denotes the Hubble parameter in conformal time and the
growth rate $f$ is defined as

$\displaystyle f(k,z)\equiv\frac{d\ln D_{1}(k,a)}{d\ln a}\,.$ (4)

Under this assumption, the velocity transfer function evolves at late time as

$\displaystyle
T_{V}(k,z)=\frac{\mathcal{H}(z)f(k,z)D_{1}(k,z)}{\mathcal{H}(z_{*})f(z_{*})D_{1}(z_{*})}T_{V}(k,z_{*})\,.$
(5)

If the continuity equation is not valid, this standard parameterization breaks
down. This would happen for example if dark matter exchanges energy with dark
energy (Pourtsidou et al., 2013). Since we want here to be agnostic about the
theory of gravity and the behaviour of dark matter, we parameterize instead
the growth of velocity at late time with a new function $G(k,z)$ such that

$\displaystyle
T_{V}(k,z)=\frac{\mathcal{H}(z)G(k,z)}{\mathcal{H}(z_{*})f(z_{*})D_{1}(z_{*})}T_{V}(k,z_{*})\,.$
(6)

Finally, the potential transfer function defined through
$\Psi(\mathbf{k},z)=T_{\Psi}(k,z)\Psi_{\rm in}(\mathbf{k})$ can be related to
the density transfer function, using the Poisson equation and assuming that
there is no anisotropic stress via

$\displaystyle
T_{\Psi}(k,z)=-\frac{3}{2}\Omega_{m}(z)\left(\frac{\mathcal{H}(z)}{k}\right)^{2}T_{\delta}(k,z)\,.$
(7)

Under these assumptions, the gravitational potential at late time evolves as

$\displaystyle
T_{\Psi}(k,z)=\frac{\mathcal{H}^{2}(z)\Omega_{m}(z)D_{1}(k,z)}{\mathcal{H}^{2}(z_{*})D_{1}(z_{*})}T_{\Psi}(k,z_{*})\,,$
(8)

where we have used that $\Omega_{m}(z_{*})=1$. Again, in order to be agnostic
about the theory of gravity, we replace this standard evolution with a new
free function $I(k,z)$ which encodes the evolution of the gravitational
potential $\Psi$ at late time:

$\displaystyle
T_{\Psi}(k,z)=\frac{\mathcal{H}^{2}(z)I(k,z)}{\mathcal{H}^{2}(z_{*})D_{1}(z_{*})}T_{\Psi}(k,z_{*})\,.$
(9)

The functions $D_{1}(k,z)$ and $G(k,z)$ have been measured via redshift-space
distortions. The function $I(k,z)$ on the other hand has never been measured
at cosmological scales. The goal of this paper is to forecast the precision
with which one can measure this function with the coming generation of
surveys.

##  3 Correlation function

Redshift surveys map the distribution of galaxies in the sky and measure the
galaxy number counts fluctuations

$\Delta\equiv\frac{N(\mathbf{n},z)-\bar{N}(z)}{\bar{N}(z)}\,,$ (10)

where $N$ is the number of galaxies per pixel detected in direction
$\mathbf{n}$ and at redshift $z$, and $\bar{N}$ denotes the average number of
galaxies per pixel at redshift $z$. In the linear regime, the dominant
contributions to the observable $\Delta$ are given by (Bonvin & Durrer, 2011;
Challinor & Lewis, 2011; Yoo et al., 2009)

$\displaystyle\Delta(\mathbf{n},z)$
$\displaystyle=b\,\delta-\frac{1}{\mathcal{H}}\partial_{r}(\mathbf{V}\cdot\mathbf{n})+\frac{1}{\mathcal{H}}\partial_{r}\Psi+\frac{1}{\mathcal{H}}\dot{\mathbf{V}}\cdot\mathbf{n}$
(11)
$\displaystyle+\left(1-5s+\frac{5s-2}{r\mathcal{H}}-\frac{\dot{\mathcal{H}}}{\mathcal{H}^{2}}+f^{\rm
evol}\right)\mathbf{V}\cdot\mathbf{n}\,,$

where $r=r(z)$ is the comoving distance to redshift $z$ and a dot denotes
derivative with respect to conformal time. The functions $b(z)$, $s(z)$ and
$f^{\rm evol}(z)$ are the galaxy bias, the magnification bias and the
evolution bias of the population under consideration.

The two terms in the first line of Eq. (11) are the standard density and
redshift-space distortions contributions. The third term is the contribution
from gravitational redshift, which depends directly on $\Psi$ and is the
target of our study. The last two terms are Doppler contributions 222Note that
other relativistic distortions contribute to $\Delta$ (Bonvin & Durrer, 2011;
Challinor & Lewis, 2011; Yoo et al., 2009), but their impact on the multipoles
of the correlation function is negligible (Jelic-Cizmek et al., 2021).
Gravitational lensing also contributes to $\Delta$, but its impact becomes
important only at large redshifts (Jelic-Cizmek et al., 2021; Lepori et al.,
2021), where we will see that $I$ cannot be measured anymore..

Density and RSD generate a monopole, quadrupole and hexadecapole in the
correlation function, which have been measured with various surveys, see e.g.
Alam et al. (2021b). The contributions from gravitational redshift and Doppler
effects on these multipoles are strongly subdominant and can safely be
neglected (Jelic-Cizmek et al., 2021). However, these relativistic effects
generate a dipole in the correlation function, which can be measured by
correlating two populations of galaxies, for example bright and faint galaxies
(Bonvin et al., 2014). We now write the multipoles in terms of the free
functions $D_{1},G$ and $I$. The even multipoles take the form

$\displaystyle\xi_{0}(z,d)$
$\displaystyle=\left[\hat{b}^{2}(z)+\frac{2}{3}\hat{b}(z)\hat{G}(z)+\frac{1}{5}\hat{G}^{2}(z)\right]\mu_{0}(z_{*},d)\,,$
$\displaystyle\xi_{2}(z,d)$
$\displaystyle=-\left[\frac{4}{3}\hat{b}(z)\hat{G}(z)+\frac{4}{7}\hat{G}^{2}(z)\right]\,\mu_{2}(z_{*},d)\,,$
$\displaystyle\xi_{4}(z,d)$
$\displaystyle=\frac{8}{35}\hat{G}^{2}(z)\,\mu_{4}(z_{*},d)\,,$ (12)

where

$\displaystyle\hat{b}(z)$ $\displaystyle\equiv b(z)\sigma_{8}(z)\,,$ (13)
$\displaystyle\hat{G}(z)$ $\displaystyle\equiv
G(z)\frac{\sigma_{8}(z)}{D_{1}(z)}=G(z)\frac{\sigma_{8}(z_{*})}{D_{1}(z_{*})}\,,$
(14)

and

$\displaystyle\mu_{\ell}(z_{*},d)$ $\displaystyle=\frac{1}{2\pi^{2}}\int
dkk^{2}\frac{P_{\delta\delta}(k,z_{*})}{\sigma_{8}^{2}(z_{*})}j_{\ell}(kd)\,.$
(15)

The functions $\mu_{\ell}(z_{*},d)$ depend on the matter power spectrum at
$z=z_{*}$, well in the matter era, and are therefore determined by physics in
the early Universe, which is tightly constrained by CMB observations (Aghanim
et al., 2020). These functions are therefore considered as fixed. Measuring
the even multipoles provides consequently direct measurements of the bias,
$\hat{b}(z)$, and of the function $\hat{G}(z)$ (Alam et al., 2021b) (which
reduces to $f(z)\sigma_{8}(z)$ under the assumption that the continuity
equation is valid.)

The dipole in the cross-correlation of bright and faint galaxies reads 333Note
that the dipole is also affected by wide-angle effects, which generate a
contribution of the form
$-2/5(\hat{b}_{\textrm{B}}-\hat{b}_{\textrm{F}})\hat{G}d/r\mu_{2}(z_{*},d)$.
This contribution can however be removed by constructing an appropriate
estimator, as has been shown in Bonvin et al. (2014); Hall & Bonvin (2017);
Bonvin & Fleury (2018). For simplicity we have set here the evolution biases
to zero $f^{\rm evol}_{\textrm{B}}=f^{\rm evol}_{\textrm{F}}$. In practice
these parameters will be measured from the galaxy populations, see e.g. Wang
et al. (2020).

$\displaystyle\xi_{1}(z,d)$
$\displaystyle=\frac{\mathcal{H}}{\mathcal{H}_{0}}\Bigg{\\{}3\left(\frac{1}{r\mathcal{H}}-1\right)\big{(}s_{\textrm{B}}(z)-s_{\textrm{F}}(z)\big{)}\hat{G}^{2}(z)$
$\displaystyle\quad+5\left(\frac{1}{r\mathcal{H}}-1\right)\left(\hat{b}_{\textrm{F}}(z)s_{\textrm{B}}(z)-\hat{b}_{\textrm{B}}(z)s_{\textrm{F}}(z)\right)\hat{G}(z)$
$\displaystyle\quad+\left(\hat{b}_{\textrm{B}}(z)-\hat{b}_{\textrm{F}}(z)\right)\left[\left(\frac{2}{r\mathcal{H}}-1\right)\hat{G}(z)-\frac{\dot{\hat{G}}(z)}{\mathcal{H}}\right]$
$\displaystyle\quad+\frac{3}{2}\left(\hat{b}_{\textrm{B}}(z)-\hat{b}_{\textrm{F}}(z)\right)\hat{I}(z)\Bigg{\\}}\,\nu_{1}(z_{*},d)\,,$
(16)

where

$\displaystyle\hat{I}(z)$ $\displaystyle\equiv
I(z)\frac{\sigma_{8}(z)}{D_{1}(z)}=I(z)\frac{\sigma_{8}(z_{*})}{D_{1}(z_{*})}\,,$
(17)

and

$\displaystyle\nu_{1}(z_{*},d)$
$\displaystyle=\frac{\mathcal{H}_{0}}{2\pi^{2}}\int
dkk\frac{P_{\delta\delta}(k,z_{*})}{\sigma_{8}^{2}(z_{*})}j_{1}(kd)\,.$ (18)

As before, the function $\nu_{1}(z_{*},d)$ which depends on $z_{*}$ is
considered fixed. Combining the dipole with the even multipoles provides a way
of measuring directly the function $\hat{I}(z)$, which encodes the evolution
of the gravitational potential $\Psi$.

Note that in our derivation we have assumed that the $k$-dependence of
$D_{1},G$ and $I$ is negligible, so that we can take them out of the integrals
in Eqs. (15) and (18). This is a common assumption, that is used in many
analyses, see e.g. Alam et al. (2021b), and that is motivated by the fact that
in the quasi-static approximation the $k$-dependence can often be neglected
(Gleyzes et al., 2016; Raveri et al., 2021). However, for some theories of
gravity this assumption may not be valid (Baker et al., 2014). Relaxing it
would slightly complicate the forecasts, but would not invalidate our method.

##  4 Forecasts

We now forecast how well we can measure $\hat{I}$ with a survey like SKA phase
2, using the Fisher formalism. SKA2 will observe close to a billion galaxies
from $z=0.1$ to $z=2$. We use the specifications of Bull (2016), for the
number density and volume, and the cosmology is fixed to the latest Aghanim et
al. (2020) values. We choose $z_{*}=10$, well in the matter era, and as
described in Section 3 we assumed that the functions $\mu_{\ell}(z_{*},d)$ and
$\nu_{1}(z_{*},d)$ are fixed by CMB constraints. Therefore we vary only
$\hat{G}$, $\hat{I}$ and the biases of the bright and faint populations in
each redshift bin 444Note that the derivatives in the Fisher matrix can be
computed analytically for our set of parameters.. We split the galaxies into a
bright and faint population with same number of galaxies per redshift bin. We
model the bias for each of the population as

$\displaystyle b_{\textrm{B}}=c_{\textrm{B}}\exp(d_{\textrm{B}}z)+\Delta
b/2\quad\mbox{and}\quad
b_{\textrm{F}}=c_{\textrm{F}}\exp(d_{\textrm{F}}z)-\Delta b/2$ (19)

involving four parameters with values $c_{\textrm{B}}=c_{\textrm{F}}=0.554$
and $d_{\textrm{B}}=d_{\textrm{F}}=0.783$ (Bull, 2016), and a bias difference
$\Delta b=1$, similar to what has been measured for BOSS in Gaztanaga et al.
(2017). We then let the biases $\hat{b}_{\textrm{B}}=b_{\textrm{B}}\sigma_{8}$
and $\hat{b}_{\textrm{F}}=b_{\textrm{F}}\sigma_{8}$ vary freely in each
redshift bins and marginalise over them. For the magnification bias, we use
the model developed in Castello et al. (2022), and we assume that the
evolution bias vanishes for the two populations. Once data will be available
the magnification bias and the evolution bias of the two populations will be
directly measurable, see e.g. Wang et al. (2020). We include shot noise and
cosmic variance in the variance of the multipoles (see Appendix C of Bonvin &
Fleury (2018)) and account for cross-correlations between different
multipoles.

We perform two different forecasts: the first one where we consider as free
parameters the functions $\hat{G}$ and $\hat{I}$ in each redshift bin, and a
second one where we parameterize the evolution of these functions with
redshift.

###  4.1 Constraints per redshift bin

Table 1: $1\sigma$ constraints on $\hat{I}$ and $\hat{G}$ relative to their corresponding fiducial value, marginalised over the bias parameters. We show the results for two values of the minimal separation $d_{\rm min}$, in Mpc/$h$. |  $z$ |  0.35 |  0.45 |  0.55 |  0.65 |  0.75 |  0.85 |  0.95 |  1.05
---|---|---|---|---|---|---|---|---|---
$d_{\rm min}$ |  $\hat{I}$ |  0.23 |  0.24 |  0.28 |  0.33 |  0.39 |  0.48 |  0.60 |  0.77
20 |  $\hat{G}$ |  0.002 |  0.002 |  0.003 |  0.003 |  0.003 |  0.004 |  0.004 |  0.005
$d_{\rm min}$ |  $\hat{I}$ |  0.27 |  0.28 |  0.32 |  0.37 |  0.45 |  0.55 |  0.69 |  0.87
32 |  $\hat{G}$ |  0.004 |  0.004 |  0.005 |  0.006 |  0.006 |  0.007 |  0.008 |  0.009

From Eq. (16), we see that the dipole depends not only on $\hat{G}$ in each
redshift bin, but also on the time derivative of $\hat{G}$. Since we do not
want to assume any evolution for $\hat{G}$, we express this time derivative in
terms of $\hat{G}$ in the neighbouring bins. We use the five-point stencil
method:

$\displaystyle\dot{\hat{G}}(z_{i})=-(1+z_{i})\mathcal{H}(z_{i})\frac{d\hat{G}(z_{i})}{dz_{i}}=-\frac{(1+z_{i})\mathcal{H}(z_{i})}{12\Delta
z}\times$
$\displaystyle\Bigg{[}-G(z_{i+2})+8G(z_{i+1})-8G(z_{i-1})+G(z_{i-2})\Bigg{]}\,.$
(20)

We have checked that for redshift bins of size $\Delta z=0.1$, the five-point
stencil method allows us to reconstruct $\dot{\hat{G}}(z_{i})$ with a
precision of $0.1$ percent. On the other hand, if we use only two bins to
reconstruct the time derivative we would make an error of up to 129 percent at
low redshift, due to the fact that $\dot{\hat{G}}$ changes sign. Such a large
mistake in $\dot{\hat{G}}$ may bias the measurement of $\hat{I}$.

In Table 1, we show the 1$\sigma$ constraints on $\hat{I}$ and $\hat{G}$,
marginalised over the bias parameters. Note that due to the five-point stencil
method we cannot constrain $\hat{I}$ in the first two and in the last two
redshift bins, since in these four bins $\dot{\hat{G}}$ is not constrained. We
show therefore the constraints starting at $z=0.35$. We compute the
constraints starting at two different minimum separations: $d_{\rm
min}=20\,\mathrm{Mpc}/h$ and $d_{\rm min}=32\,\mathrm{Mpc}/h$, since non-
linearities have been shown to become relevant around those scales (Bonvin et
al., 2020). Since the signal-to-noise ratio peaks around 30 Mpc$/h$ and then
slowly decreases with separation (see e.g. Fig. 5 in Bonvin & Fleury (2018)),
separations above $d_{\rm max}=160\,\mathrm{Mpc}/h$ are irrelevant. The
constraints are therefore obtained from separations well inside the horizon,
which justifies the use of the quasi-static approximation.

Figure 1: Joint constraints of $\hat{I}$ and $\hat{G}$ at $z=0.35$, with
$d_{\rm min}=20\,\mathrm{Mpc/}h$.

We see that at low redshift, the constraints on $\hat{I}$ are very good,
providing a direct measurement on the evolution of the potential $\Psi$ with a
precision of 20-30 percent. Combining such a measurement with gravitational
lensing will allow us to distinguish unambiguously between deviations from GR
and dark fifth force models. Comparing with current tests of gravity, we see
that the precision with which $\Psi$ will be measured with the coming
generation of survey is similar to the precision with which the evolution of
the velocity field is currently measured: in Alam et al. (2021b), for example,
the combination $f(z)\sigma_{8}(z)$ is measured with a precision of 10-30
percent in various redshift bins. The future constraints on $\hat{G}$ are
tighter than those on $\hat{I}$ by almost two order of magnitude. This is not
surprising since $\hat{G}$ is constrained by the even multipoles, which will
be measured with exquisite accuracy with SKA2, whereas $\hat{I}$ is
constrained only by the dipole, which has a significantly lower signal-to-
noise ratio. Interestingly, the constraints on $\hat{I}$ are only mildly
degraded (by $14-18\%$) when increasing $d_{\rm min}$ from 20 to 32 Mpc/$h$.
In contrast, the constraints on $\hat{G}$ are degraded by 100%. This is due to
the fact that the gravitational potential is much less affected by non-
linearities than density and RSD.

In Fig. 1 we show the joint constraints on $\hat{G}$ and $\hat{I}$ at
$z=0.35$. We see that the parameters are almost not degenerate. This is due to
the fact that the even multipoles are only sensitive to $\hat{G}$ and can
therefore efficiently break the degeneracy between $\hat{G}$ and $\hat{I}$ in
the dipole.

Finally, we have also performed forecasts for the parameter
$\tilde{I}\equiv(\hat{b}_{\textrm{B}}-\hat{b}_{\textrm{F}})\hat{I}$ which
directly enters into the dipole. The constraints on this parameter are only
very slightly better than those on $\hat{I}$ (by 0.1 percent). Hence even
though $\hat{I}$ is completely degenerate with the biases in the dipole, these
biases are so well constrained by the even multipoles, that the degeneracy is
completely broken.

###  4.2 Constraints from a specific time evolution

When constraining deviations from GR, standard RSD analyses usually assume a
specific time evolution for the functions encoding these deviations. One
common assumption is to use that the deviations evolve proportionally to the
amount of dark energy $\Omega_{\Lambda}(z)$, see e.g Alam et al. (2021a). Here
we consider the two following models for the evolution of $\hat{G}$ and
$\hat{I}$:

$\displaystyle\hat{G}(z)=$ $\displaystyle
f(z)\sigma_{8}(z)\left[1+\hat{G}_{0}X(z)\right]\,,$ (21)
$\displaystyle\hat{I}(z)=$
$\displaystyle\Omega_{m}(z)\sigma_{8}(z)\left[1+\hat{I}_{0}X(x)\right]\,,$
(22)

with $X(z)=1$ for $z\in[0,2]$ in the first model and
$X(z)=\Omega_{\Lambda}(z)/\Omega_{\Lambda}(z=0)$ in the second model. With
this, the analysis is significantly simplified since $\dot{\hat{G}}$ is
directly determined by Eq. (21). Moreover in this case, instead of considering
one free bias parameter per redshift for each population, we assume that the
biases evolve as in Eq. (19), with four free parameters
$c_{\textrm{B}},c_{\textrm{F}},d_{\textrm{B}}$ and $d_{\textrm{F}}$.

The marginalised constraints on $\hat{G}_{0}$ and $\hat{I}_{0}$ are shown in
Table 2. For the constant model, we reach a precision of 10% on $\hat{I}_{0}$,
which decreases to 15% for the dynamical model.

Table 2: $1\sigma$ constraints on $\hat{I}_{0}$ and $\hat{G}_{0}$, marginalised over the bias parameters, for the two models described after Eq. (22). | No evolution | Evolution
---|---|---
$d_{\rm min}[{\rm Mpc}/h]$ | 20 | 32 | 20 | 32
$\hat{I}_{0}$ | 0.10 | 0.11 | 0.15 | 0.18
$\hat{G}_{0}$ | 0.001 | 0.002 | 0.002 | 0.003

##  5 Conclusion

In this Letter we have shown that the coming generation of galaxy surveys,
like SKA2, will be able to measure directly the evolution of the gravitational
potential, $\Psi$, with an accuracy of 10–30%. Such a measurement will be a
game changer to distinguish modified theories of gravity from models with a
dark fifth force, since it is fully complementary to current measurements that
are sensitive to the matter density and velocity, and to the sum of the two
gravitational potentials.

One difficulty in measuring the evolution of $\Psi$ is that it is degenerate
with the time derivative of the velocity, which is itself unknown. However, we
have shown that since future surveys will be able to measure the velocity in
thin redshift bins, this can be used to reconstruct, in a model-independent
way, the time derivative of the velocity and consequently break the degeneracy
with $\Psi$.

Comparing our forecasts with current constraints on modified gravity, we see
that future constraints on $\Psi$ are roughly of the same order of magnitude
as current constraints on the velocity $V$, and worse by two orders of
magnitude than future constraints on $V$. This is simply due to the fact that
the dipole (which is constraining $\Psi$) has a signal-to-noise ratio which is
significantly lower than that of the even multipoles (which are constraining
$V$). However, it is worth mentioning that the very tight constraints expected
on $V$ do not directly translate into tight constraints on modified gravity
parameters, like for example $\mu_{0}$ (which encodes a modification to
Poisson equation (Pogosian et al., 2010)). As shown in Castello et al. (2022),
if one does not assume that dark matter obeys the weak equivalence principle,
$\mu_{0}$ can only be constrained with a precision of 15% with a survey like
SKA2, which is very similar to the constraints we found here on $\Psi$. This
seems therefore to be the level at which modifications to GR can be tested in
a truly model-independent way.

A first application of this novel measurement of $\Psi$, will be to use it in
combination with gravitational lensing to measure the anisotropic stress in a
model-independent way.

## Acknowledgements

This project has received funding from the Swiss National Science Foundation
and from the European Research Council (ERC) under the European Union’s
Horizon 2020 research and innovation program (Grant agreement No. 863929;
project title “Testing the law of gravity with novel large-scale structure
observables"). We acknowledge the use of the fftlog-python code written by
Goran Jelic-Cizmek and available at https://github.com/JCGoran/fftlog-python.

## Data Availability

No new data were generated in support of this research. The numerical results
underlying this article will be shared on reasonable request to the
corresponding author.

## References

  * Abbott et al. (2022) Abbott T. M. C., et al., 2022, Phys. Rev. D, 105, 023520
  * Adams (1925) Adams W. S., 1925, The Observatory, 48, 337
  * Aghanim et al. (2020) Aghanim N., et al., 2020, Astron. Astrophys., 641, A6
  * Alam et al. (2017) Alam S., Zhu H., Croft R. A. C., Ho S., Giusarma E., Schneider D. P., 2017, Mon. Not. Roy. Astron. Soc., 470, 2822
  * Alam et al. (2021a) Alam S., et al., 2021a, JCAP, 11, 050
  * Alam et al. (2021b) Alam S., et al., 2021b, Phys. Rev. D, 103, 083533
  * Amendola et al. (2013) Amendola L., et al., 2013, Living Rev. Rel., 16, 6
  * Amendola et al. (2018) Amendola L., et al., 2018, Living Rev. Rel., 21, 2
  * Archidiacono et al. (2022) Archidiacono M., Castorina E., Redigolo D., Salvioni E., 2022
  * Asghari et al. (2019) Asghari M., Beltrán Jiménez J., Khosravi S., Mota D. F., 2019, JCAP, 04, 042
  * Baker et al. (2014) Baker T., Ferreira P. G., Leonard C. D., Motta M., 2014, Phys. Rev. D, 90, 124030
  * Barros et al. (2019) Barros B. J., Amendola L., Barreiro T., Nunes N. J., 2019, JCAP, 01, 007
  * Beutler & Di Dio (2020) Beutler F., Di Dio E., 2020, JCAP, 07, 048
  * Bonvin (2014) Bonvin C., 2014, Class. Quant. Grav., 31, 234002
  * Bonvin & Durrer (2011) Bonvin C., Durrer R., 2011, Phys. Rev. D, 84, 063505
  * Bonvin & Fleury (2018) Bonvin C., Fleury P., 2018, JCAP, 05, 061
  * Bonvin & Pogosian (2022) Bonvin C., Pogosian L., 2022, Modified Einstein versus Modified Euler for Dark Matter (arXiv:2209.03614)
  * Bonvin et al. (2014) Bonvin C., Hui L., Gaztanaga E., 2014, Phys. Rev. D, 89, 083535
  * Bonvin et al. (2016) Bonvin C., Hui L., Gaztanaga E., 2016, JCAP, 08, 021
  * Bonvin et al. (2020) Bonvin C., Franco F. O., Fleury P., 2020, JCAP, 08, 004
  * Bull (2016) Bull P., 2016, Astrophys. J., 817, 26
  * Castello et al. (2022) Castello S., Grimm N., Bonvin C., 2022
  * Challinor & Lewis (2011) Challinor A., Lewis A., 2011, Phys. Rev. D, 84, 043516
  * Croft (2013) Croft R. A. C., 2013, Mon. Not. Roy. Astron. Soc., 434, 3008
  * Einstein (1916) Einstein A., 1916, Annalen Phys., 49, 769
  * Frieman & Gradwohl (1991) Frieman J. A., Gradwohl B.-A., 1991, Phys. Rev. Lett., 67, 2926
  * Gaztanaga et al. (2017) Gaztanaga E., Bonvin C., Hui L., 2017, JCAP, 01, 032
  * Gleyzes et al. (2015) Gleyzes J., Langlois D., Mancarella M., Vernizzi F., 2015, JCAP, 1508, 054
  * Gleyzes et al. (2016) Gleyzes J., Langlois D., Mancarella M., Vernizzi F., 2016, JCAP, 02, 056
  * Gradwohl & Frieman (1992) Gradwohl B.-A., Frieman J. A., 1992, ApJ, 398, 407
  * Greenstein & Trimble (1967) Greenstein J. L., Trimble V., 1967, AJ, 72, 301
  * Hall & Bonvin (2017) Hall A., Bonvin C., 2017, Phys. Rev. D, 95, 043530
  * Hui et al. (2009) Hui L., Nicolis A., Stubbs C., 2009, Phys. Rev. D, 80, 104002
  * Jelic-Cizmek et al. (2021) Jelic-Cizmek G., Lepori F., Bonvin C., Durrer R., 2021, JCAP, 04, 055
  * Joyce et al. (2016) Joyce A., Lombriser L., Schmidt F., 2016, Ann. Rev. Nucl. Part. Sci., 66, 95
  * Köhlinger et al. (2017) Köhlinger F., et al., 2017, Mon. Not. Roy. Astron. Soc., 471, 4412
  * Koyama (2016) Koyama K., 2016, Rept. Prog. Phys., 79, 046902
  * Lepori et al. (2021) Lepori F., et al., 2021
  * Motta et al. (2013) Motta M., Sawicki I., Saltas I. D., Amendola L., Kunz M., 2013, Phys. Rev. D, 88, 124035
  * Pogosian et al. (2010) Pogosian L., Silvestri A., Koyama K., Zhao G.-B., 2010, Phys. Rev. D, 81, 104023
  * Pound & Snider (1965) Pound R. V., Snider J. L., 1965, Phys. Rev., 140, B788
  * Pourtsidou et al. (2013) Pourtsidou A., Skordis C., Copeland E. J., 2013, Phys. Rev. D, 88, 083505
  * Raveri et al. (2021) Raveri M., et al., 2021, arXiv e-prints
  * Sadeh et al. (2015) Sadeh I., Feng L. L., Lahav O., 2015, Phys. Rev. Lett., 114, 071103
  * Saga et al. (2022) Saga S., Taruya A., Rasera Y., Breton M.-A., 2022, Mon. Not. Roy. Astron. Soc., 511, 2732
  * Saltas et al. (2014) Saltas I. D., Sawicki I., Amendola L., Kunz M., 2014, Phys. Rev. Lett., 113, 191101
  * Sobral-Blanco & Bonvin (2021) Sobral-Blanco D., Bonvin C., 2021, Phys. Rev. D, 104, 063516
  * Song et al. (2010) Song Y.-S., Hollenstein L., Caldera-Cabral G., Koyama K., 2010, JCAP, 04, 018
  * Song et al. (2011) Song Y.-S., Zhao G.-B., Bacon D., Koyama K., Nichol R. C., Pogosian L., 2011, Phys. Rev. D, 84, 083523
  * Vessot et al. (1980) Vessot R. F. C., et al., 1980, Phys. Rev. Lett., 45, 2081
  * Wang et al. (2020) Wang M. S., Beutler F., Bacon D., 2020, Mon. Not. Roy. Astron. Soc., 499, 2598
  * Wojtak et al. (2011) Wojtak R., Hansen S. H., Hjorth J., 2011, Nature, 477, 567
  * Yoo et al. (2009) Yoo J., Fitzpatrick A. L., Zaldarriaga M., 2009, Phys. Rev. D, 80, 083514
  * Yoo et al. (2012) Yoo J., Hamaus N., Seljak U., Zaldarriaga M., 2012, Phys. Rev. D, 86, 063514