# Exploring speech style spaces with language models:
Emotional TTS without emotion labels

###### Abstract

Many frameworks for emotional text-to-speech (E-TTS) rely on human-annotated
emotion labels that are often inaccurate and difficult to obtain. Learning
emotional prosody implicitly presents a tough challenge due to the subjective
nature of emotions. In this study, we propose a novel approach that leverages
text awareness to acquire emotional styles without the need for explicit
emotion labels or text prompts. We present TEMOTTS, a two-stage framework for
E-TTS that is trained without emotion labels and is capable of inference
without auxiliary inputs. Our proposed method performs knowledge transfer
between the linguistic space learned by BERT and the emotional style space
constructed by global style tokens. Our experimental results demonstrate the
effectiveness of our proposed framework, showcasing improvements in emotional
accuracy and naturalness. This is one of the first studies to leverage the
emotional correlation between spoken content and expressive delivery for
emotional TTS.111Demo page: https://kodhandarama.github.io/TEMOTTSdemo

##  1 Introduction

Recent neural text-to-speech (TTS) models [1, 2, 3] have demonstrated the
capability to generate high-quality speech when provided with a large amount
of high-quality data, allowing for satisfactory alignments between speech and
text. As we aim to bridge the gap between human speech and synthesized speech,
we recognize that prosody modeling has a huge role to play in the synthesis of
natural and expressive speech [4]. Prosody modeling in deep neural TTS models
has been achieved by predicting low-level features such as duration, pitch,
energy, and emphasis. However, the synthesis of emotional speech remains a
challenging task due to the highly complex nature of emotions.

The goal of E-TTS systems is to effectively capture the prosodic aspects of
speech, enabling the accurate conveyance of emotion-specific paralinguistic
information to the listener. Emotions are highly stochastic functions of low-
level prosody features - which makes it hard to explicitly model emotional
prosody. This task is also constrained by the limited availability of emotion-
annotated datasets. E-TTS systems have the added benefit of being emotion-
aware compared to expressive TTS models, thereby providing the opportunity for
new applications like personalized virtual assistants, interactive
storytelling systems, and emotive AI therapists.

Previous approaches in E-TTS have predominantly relied on emotion labels to
condition models for learning emotional speaking styles. Early approaches used
one-hot encoded emotion embeddings during training [5], while later work
popularized the use of global style tokens (GST) [5]. Various GST-based
methods such as centroid-based weights [6], interpolation algorithms [7], and
joint style-linguistic emotion embeddings [8] were applied to generate
emotion-specific styles. These methods, however, encourage the style encoder
to learn fixed style embeddings, thereby limiting exploration of the style
space. In [9], reinforcement learning is used to interact with an emotion
recognizer to improve discriminability in the GST weight distribution. While
the aforementioned approaches were successful in generating emotionally
expressive speech of high quality, they were heavily reliant on emotion-
labeled datasets. In this paper, we extend the exploration of GST in E-TTS
without the reliance on explicit emotion labels.

Figure 1: Proposed TEMOTTS framework with stage I and stage II. Red modules
contain trainable weights and purple modules contain fixed weights; CE loss
represents Categorical Cross Entropy loss.

Many E-TTS frameworks have been trained on acted datasets where the spoken
content and intended emotion are mismatched [10]. Consequently, these models
often struggle to capture conversational speaking styles effectively.
Furthermore, several E-TTS models [11, 12] require additional inputs such as
reference audio and/or emotion labels during inference, resulting in speech
that may not faithfully convey the emotional subtleties present in the input
text. This limitation affects the scalability and significantly reduces the
potential use cases of the TTS system. To address these challenges, our
framework learns to capture and generate appropriate emotions solely based on
the inherent emotional characteristics of the language, thus enhancing the
system’s robustness and adaptability.

In real life, our spoken words and affective prosody have a natural
correlation, but this aspect is often overlooked when constructing TTS
datasets and models. With the increasing capabilities of language models [13,
14] to understand deep semantic information from text, we are motivated to use
this text awareness to connect the emotional context in the text to emotional
prosody in speech.

In this paper, we propose an E-TTS framework based on FastSpeech2, trained in
two stages to directly learn emotion representations from emotion-unlabeled
training data. We utilize style tokens to construct an emotional style space
and a fine-tuned BERT model to explore this space effectively. The main
contributions of the paper are summarized as follows:

  * •

We present a novel two-stage training for an E-TTS model, eliminating the
necessity for emotion labels during training;

  * •

We establish emotional text-awareness by establishing a direct connection
between the linguistic and affective spaces using a language model; and

  * •

We present an inference strategy that enables the accurate synthesis of
emotional speech, reflecting the emotion embedded within the text, all without
the requirement for any auxiliary inputs. This is one of the first works to
harness the emotional interconnection between our spoken content and their
emotive delivery - for emotional speech synthesis.

##  2 Related work

###  2.1 Towards using fewer emotion labels for E-TTS

The scarcity of extensive emotion-labeled datasets poses significant
limitations on E-TTS research. This limitation arises due to the high cost of
labeling emotions for the extensive data required by TTS models [15]. The
demand for high-quality speech data in TTS restricts the direct utilization of
datasets from speech emotion recognition (SER) research [16]. In light of
these challenges, we are motivated to explore other ways to obtain emotional
information to condition the TTS model. This problem of reducing the number of
emotion annotations has been addressed through various approaches. In [17], a
semi-supervised method was employed to enhance the interpretability of GST
tokens in E-TTS. Similarly, in [18], an expressive dataset was leveraged to
bolster the training of their graph neural network-based framework,
integrating semi-supervised learning. Furthermore, [12] introduced a cross-
domain SER that predicted soft labels for the TTS dataset and utilized these
labels to condition the GST style token weights.

In this paper, we omit the use of emotion labels and, instead, emphasize a
text-aware approach that leverages the inherent emotional connection within
audio-text pairs.

###  2.2 Text-aware TTS

Beyond employing texts solely for learning alignments, exploring their
utilization for acquiring prosodic representations constitutes an intriguing
subfield in TTS research. Several expressive TTS frameworks, such as TP-GST
[19] and MsEmoTTS [20] have explored text-awareness by learning
representations for linguistic prosody. EMSpeech [21], on the other hand,
directly aims to predict emotion labels from the text at inference by training
on a dataset with human-labeled emotion annotations.

Incorporating language models like BERT [22] into TTS has been another avenue
of research. These models have been used to enhance prosody [23, 24] and learn
emotional representations to condition the TTS model [25, 26]. Recent models
have also attempted to reduce the modality gap between style text inputs and
reference speech by aligning the two modalities during training [27, 10].
InstructTTS [28], on the other hand, investigates text-awareness in two roles:
as an input and as a style tag. With the increasing popularity of prompt
encoders in TTS [29], there is also a growing need for prompt-labeled datasets
that need laborious manual annotations.

More recently, GPT-3 [14] has been utilized to train a language model-based
emotion prediction network [30] for E-TTS. In contrast to previous studies
that focused on training emotional speech synthesis using labeled data and
incorporating auxiliary inputs during inference, our goal is to address the
clear research gap between models that require these specific conditions for
emotional speech synthesis and expressive TTS models that can operate without
such constraints. We believe that language model based emotion prediction
represents the future direction for advancing E-TTS systems. This aspect
stands as a central focus of our paper.

##  3 TEMOTTS

We propose TEMOTTS, a text-aware E-TTS framework designed to directly acquire
emotional speaking styles from textual input. Our choice of Fastspeech2 as the
backbone is motivated by its explicit modeling of pitch, duration, and energy,
resulting in the generation of expressive speech, as well as its efficiency in
terms of both training and inference due to parallelism. It is worth noting
that the underlying concept of TEMOTTS can be implemented within any encoder-
decoder framework.

TEMOTTS comprises two training stages: Stage I constructs a style space using
GST and trains Fastspeech2 modules. In Stage II, an adaptation module maps
emotional text embeddings from a language model to corresponding emotional
style representations in the style space. We note that multi-step training
strategies are shown to be effective in modeling alignment and prosody-related
information separately [4]. Since large, emotion-labelled data is hard to
acquire, multi-step approaches allow for the independent modeling of the two
tasks [31, 32]. Our proposed two-stage training strategy is inspired by these
methods, which have proven to be effective in addressing prosody-related
challenges in E-TTS.

Figure 2: Visualization of emotional style space and emotional text embeddings
space with a conceptual link.

###  3.1 Stage I: Constructing an emotional style space

In Stage I, we train the acoustic modules including a text encoder, variance
adaptors, a decoder, and a GST network, as shown in Fig. 1. The encoder
converts phoneme embeddings into a phoneme-hidden sequence, while the variance
adapter introduces pitch, duration, and energy variations. The decoder then
translates this hidden sequence into the mel-spectrogram output.

The GST network, composed of a reference encoder and style embeddings,
conditions the TTS model’s encoder for various speaking styles. We
specifically use GST because it constructs a style space in an unsupervised
manner and provides a means to directly navigate this space using GST
combination weights.

Since we use emotional training data, we hypothesize that the model implicitly
learns speaking styles that are representative of emotions [7]. The style
encoder is trained to decompose the input speech into a select set of basis
vectors which can represent various speaking styles. This happens in an
unsupervised manner and helps the base FastSpeech2 framework explore the
latent space of emotions.

###  3.2 Stage II: Exploration of style space with language model

Stage II consists of three modules, as illustrated in Fig. 1:
Emotion Data Pruning: To capture emotional correlations between speech-text
pairs, it is crucial to select utterances with substantial emotional context.
To achieve this, we use a fine-tuned DistilRoBERTa-base language model
222https://huggingface.co/michellejieli/emotion_text_classifier, to predict
emotion class probabilities from text. For each example, we calculate the
dominant emotion using this model. If the probability of the dominant emotion
class exceeds an experimentally chosen threshold $P_{th}$, we include the
example in the training dataset for Stage II.
Emotion Text Embedding: To represent emotional content in text, we extract
emotion text embeddings using mean pooling over the last layer of the fine-
tuned DistilRoBERTa-base language model.
Adaptation Module: We establish an emotional correlation between emotional
context in text and emotional prosody in speech. In Fig. 2, we utilize t-SNE
[33] to visualize the emotional style space created by GST weight distribution
in the emotion data-pruned dataset, alongside the space constituted by
emotional text embeddings. A symbolic line connects two representative points
from each space, illustrating the relationship between the emotional style
space and the emotional text embedding space. Notably, Fig. 2 reveals a
clustering similarity between the spaces. Motivated by this observation, we
introduce an adaptation module capable of mapping emotional text embeddings
from the language model to their corresponding emotional style
representations. The emotion text embeddings are fed as input into this module
and the GST weights, predicted by the trained reference encoder, serve as
target values. This mapping is treated as a multi-class classification task,
optimized using a categorical cross-entropy loss function. It is worth noting
that we incorporate soft labels during training since the goal of stage II is
to predict GST weights, which inherently range from 0 to 1.

###  3.3 Inference

During inference, TEMOTTS only requires text as input, as shown in Fig. 3. The
text is processed by the language model to generate an emotional text
embedding. The adaptation layers predict GST weights, giving the model access
to the emotion style space constructed by style tokens. The style embedding,
derived from a weighted linear combination of these tokens, conditions the
phoneme embedding module in stage I to synthesize text-aware emotional speech.

Figure 3: Inference strategy of TEMOTTS.

(a) VITS (b) Fastspeech2

(c) TEMOTTS

Figure 4: SER confusion matrices of 4 emotion categories.

##  4 Experiments

In this section, we conduct both objective and subjective evaluations to
assess the performance of our proposed method, TEMOTTS, in terms of emotional
accuracy and naturalness.

We train TEMOTTS stage I using a combination of two datasets: LJSpeech [34]
and an emotional TTS dataset 333https://github.com/skit-ai/emotion-tts-
dataset[35], both sampled at 22.05kHz. The emotional TTS dataset comprises
high-quality recordings of nine emotions by an English female speaker. Unlike
most acted emotional TTS datasets, this data is spoken in a natural style and
has a high correlation between linguistic content and emotional speaking
style. Although we train TEMOTTS with a recorded emotional TTS dataset, the
model is designed to leverage any conversational speech dataset that has
sufficient emotional variation. We believe that the acquisition of such data
is more viable than manual annotation of emotional speech datasets. We
partition our dataset into training, validation, and test sets with a ratio of
18:1:1. For stage II, we use emotion data pruning described in Section 3.2 to
select a subset of the emotional dataset [35]. For evaluation, we create two
test sets using sentences generated by GPT-3 [14]. The first set comprises of
randomly generated sentences for assessing Word Error Rate (WER) and Character
Error Rate (CER). The second set consists of sentences enriched with high-
emotion content, designed to evaluate the emotional text-awareness of the
models.

###  4.1 Experimental setup

We convert the raw waveform into a mel-spectrogram, configuring the hop size
to 256 and making use of 80 mel bins. To extract phoneme duration, we utilize
Montreal Forced Alignment [36]. The adaptation module consists of eight fully
connected dense layers of sizes - 772, 600, 500, 400, 300, 200, 100, 50, 40
and 16 with ReLU activation. We follow the
ESPNet2444https://github.com/espnet/espnet framework to implement other
Fastspeech2-based modules and the reference encoder in GST network. A pre-
trained Parallel WaveGAN [37] 555https://github.com/kan-
bayashi/ParallelWaveGAN trained on LJSpeech is used as the vocoder, while the
remaining components are trained from scratch. We utilize the Adam optimizer
with a Noam scheduler, using the following parameters: $\beta_{1}$ = 0.9,
$\beta_{2}$ = 0.999, $\epsilon$ = 10e-9, 4000 warm-up steps and set the
learning rate to 10e-4. The model with the best performance on the validation
set is selected for evaluation after training.

We set the emotion data pruning parameter $P_{th}$ to 0.7 after experimenting
with values between 0.5 and 0.9.

####  4.1.1 Choice of baselines:

To ensure a fair comparison with the proposed TEMOTTS method, we consider
baselines that are trained under the same constraints -

  1. 1.

without emotion labels; and

  2. 2.

inference without any auxiliary inputs.

With the lack of such E-TTS frameworks, we consider Fastspeech2 [2] and VITS
[38] for comparison. Both FastSpeech2 and VITS are expressive TTS frameworks
that learn low-level prosody modeling from the training data. The training of
Fastspeech2 is identical to stage I of our proposed method. For VITS, we train
the model 666VITS code: https://tts.readthedocs.io/en/latest/models/vits.html
with the same data as the proposed model. We also note that using style
encoder-based TTS models would require additional inputs such as reference
speech or prompts and hence does not constitute a fair baseline.

####  4.1.2 Choice of number of style tokens:

The selection of the number of style tokens ($N_{tokens}$) within the style
encoder is a critical hyperparameter in our approach. Experimentation reveals
that increasing $N_{tokens}$ significantly reduces the model’s capacity to
generate emotionally expressive speech while setting $N_{tokens}$ too low
constrained the emotional style space. After thorough experimentation with
$N_{tokens}$ values of 8, 16, 32, and 64, we find that $N_{tokens}$ = 16
provides the best results.

###  4.2 Objective evaluation

We calculate CER and WER to assess the intelligibility of the synthesized
speech. Table 1 presents CER and WER results obtained from the Wav2Vec 2.0
Large ASR
model777https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self. These
results indicate that our proposed method, with its text-awareness, enhances
the intelligibility of synthesized speech compared to the baselines.

To assess the capability of the model to synthesize emotional speech, we train
an SER by finetuning Wave2Vec2.0 [39] on the emotional TTS dataset. We
synthesize 50 sentences each for the four emotion classes - anger, happiness,
neutral, and sadness - using GPT3 [14]. We evaluate the ability of the models
to synthesize the expected emotion from text using the trained SER. We treat
the dominant emotion present in the text as the true label. Confusion matrices
from the SER (Fig. 4) clearly illustrate that the proposed model demonstrates
a higher level of confidence in synthesizing emotional speech that conveys the
intended emotion from text, in contrast to the baseline models.

Table 1: Objective and subjective evaluation results, where MOS results have
95% confidence intervals.

Method | CER(%) | WER(%) | BWS | MOS
---|---|---|---|---
Best(%) | Worst(%)
Ground Truth | 1.68 | 7.32 | - | -
VITS[1] | 8.22 | 18.57 | 4.20 | 79.52 |  3.02 $\pm$ 0.07
Fastspeech2[2] | 6.88 | 18.34 | 14.76 | 18.57 |  3.68 $\pm$ 0.06
TEMOTTS | 4.08 | 13.11 | 80.95 | 1.90 | 3.75 $\pm$ 0.06

(a) Happy text: ”I am excited for the new football season to start!”

(b)  Sad text: ”I am so sad these days.”

Figure 5: Pitch contours of speech synthesized by TEMOTTS and Fastspeech2
along with ground truth.

####  4.2.1 Comparing pitch contours:

In Fig. 5, we plot the pitch contour of two different emotional sentences. We
clearly observe that a) TEMOTTS is capable of modeling pitch more accurately
with respect to the ground truth, and b) TEMOTTS has a higher dynamic range in
pitch. This shows the effectiveness of our model in terms of expressiveness
and ability to convey emotions accurately.

###  4.3 Subjective evaluation

We conduct listening experiments with 15 subjects to assess speech quality and
the ability to synthesize text-aware emotional speech, using a total of 90
utterances for the tests.

In the Mean Opinion Score (MOS) evaluation [1], participants rate speech
quality using a five-point scale for 15 speech samples generated by each
model, with text generation performed randomly by GPT-3[14]. As indicated in
Table. 1, the proposed TEMOTTS generates speech with a higher level of
naturalness than both baselines, achieving a MOS score of 3.75. We believe
that the ability of TEMOTTS to be emotionally aware while synthesizing speech
leads to better naturalness.

To assess the emotional text-awareness of the model, we conduct a Best-Worst
Scaling (BWS) test. We used GPT-3 to generate 15 sentences with high emotion
polarity. We then synthesize these sentences using our method and baselines,
and listeners are tasked with selecting the best and worst samples based on
how accurately the speech conveys the emotion in the text. Table. 1 shows that
TEMOTTS outperforms both baselines in terms of synthesizing text-emotion
accurate speech, receiving the highest best votes (80.95%) and the lowest
worst votes (1.9%).

In both subjective and objective evaluations, the proposed method shows the
clear advantage of being emotionally text-aware. Even though the baselines
Fastspeech2 and VITS are trained to learn appropriate low-level prosody
features for the input text, we clearly show that this does not necessarily
constitute the learning of emotional prosody. Hence, the proposed method
outperforms these baselines, especially in the cases where the spoken content
is emotional in nature.

##  5 Conclusion

In this study, we introduce TEMOTTS, an E-TTS framework that overcomes the
challenges posed by human-annotated emotion labels and the intricate nature of
emotional prosody learning. TEMOTTS relies on text awareness to acquire
emotional styles, eliminating the need for explicit emotion labels during
training and auxiliary inputs during inference. Our experiments showcase
significant improvements in emotional accuracy, naturalness, and
intelligibility compared to baselines. This work presents a novel approach to
enhancing the emotional richness of speech synthesis by connecting spoken
content and its expressive delivery.

## References

  * [1] Yuxuan Wang, R. J. Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Z. Chen, Samy Bengio, Quoc V. Le, Yannis Agiomyrgiannakis, Robert A. J. Clark, and Rif A. Saurous,  “Tacotron: Towards end-to-end speech synthesis,”  in Interspeech, 2017.
  * [2] Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu,  “Fastspeech 2: Fast and high-quality end-to-end text to speech,”  in International Conference on Learning Representations, 2021.
  * [3] Adrian Łańcucki,  “Fastpitch: Parallel text-to-speech with pitch prediction,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 6588–6592.
  * [4] Rui Liu, Berrak Sisman, Guanglai Gao, and Haizhou Li,  “Expressive tts training with frame and style reconstruction loss,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1806–1818, 2021.
  * [5] Younggun Lee, Soo-Young Lee, and Azam Rabiee,  “Emotional end-to-end neural speech synthesizer,”  in NIPS2017. Neural Information Processing Systems Foundation, 2017\.
  * [6] Ohsung Kwon, Inseon Jang, Chung Hyun Ahn, and Hong-Goo Kang,  “An effective style token weight control technique for end-to-end emotional speech synthesis,”  IEEE Signal Processing Letters, vol. 26, pp. 1383–1387, 2019.
  * [7] Seyun Um, Sangshin Oh, Kyungguen Byun, Inseon Jang, Chung Hyun Ahn, and Hong-Goo Kang,  “Emotional speech synthesis with rich and granularized control,”  ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7254–7258, 2019.
  * [8] Ohsung Kwon, Inseon Jang, Chung Hyun Ahn, and Hong-Goo Kang,  “Emotional speech synthesis based on style embedded tacotron2 framework,”  2019 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), pp. 1–4, 2019.
  * [9] Rui Liu, Berrak Sisman, and Haizhou Li,  “Reinforcement learning for emotional text-to-speech synthesis with improved emotion discriminability,”  in Interspeech, 2021.
  * [10] Yookyung Shin, Younggun Lee, Suhee Jo, Yeongtae Hwang, and Taesu Kim,  “Text-driven emotional style control and cross-speaker style transfer in neural tts,”  in Interspeech, 2022.
  * [11] Wei Zhao and Zheng Yang,  “An emotion speech synthesis method based on vits,”  Applied Sciences, vol. 13, no. 4, pp. 2225, 2023.
  * [12] Xiong Cai, Dongyang Dai, Zhiyong Wu, Xiang Li, Jingbei Li, and Helen M. Meng,  “Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition,”  ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5734–5738, 2020.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova,  “Bert: Pre-training of deep bidirectional transformers for language understanding,”  in North American Chapter of the Association for Computational Linguistics, 2019.
  * [14] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.,  “Language models are few-shot learners,”  Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  * [15] Jaehyeon Kim, Sungwon Kim, Jungil Kong, and Sungroh Yoon,  “Glow-tts: A generative flow for text-to-speech via monotonic alignment search,”  Advances in Neural Information Processing Systems, vol. 33, pp. 8067–8077, 2020.
  * [16] Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan,  “Iemocap: Interactive emotional dyadic motion capture database,”  Language resources and evaluation, vol. 42, pp. 335–359, 2008.
  * [17] Pengfei Wu, Zhenhua Ling, Lijuan Liu, Yuan Jiang, Hongchuan Wu, and Lirong Dai,  “End-to-end emotional speech synthesis using style tokens and semi-supervised training,”  in 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 2019, pp. 623–627.
  * [18] Jiaxu He, Cheng Gong, Longbiao Wang, Di Jin, Xiaobao Wang, Junhai Xu, and Jianwu Dang,  “Improve emotional speech synthesis quality by learning explicit and implicit representations with semi-supervised training,”  Proc. Interspeech 2022, pp. 5538–5542, 2022.
  * [19] Daisy Stanton, Yuxuan Wang, and R. J. Skerry-Ryan,  “Predicting expressive speaking style from text in end-to-end speech synthesis,”  2018 IEEE Spoken Language Technology Workshop (SLT), pp. 595–602, 2018.
  * [20] Shun Lei, Yixuan Zhou, Liyang Chen, Zhiyong Wu, Xixin Wu, Shiyin Kang, and Helen Meng,  “Msstyletts: Multi-scale style modeling with hierarchical context information for expressive speech synthesis,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
  * [21] Chenye Cui, Yi Ren, Jinglin Liu, Feiyang Chen, Rongjie Huang, Ming Lei, and Zhou Zhao,  “Emovie: A mandarin emotion speech dataset with a simple emotional text-to-speech model,”  in Interspeech, 2021.
  * [22] Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova,  “Bert: Pre-training of deep bidirectional transformers for language understanding,”  in Proceedings of NAACL-HLT, 2019, pp. 4171–4186.
  * [23] Liping Chen, Yan Deng, Xi Wang, Frank K. Soong, and Lei He,  “Speech bert embedding for improving prosody in neural tts,”  ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6563–6567, 2021.
  * [24] Yujia Xiao, Lei He, Huaiping Ming, and Frank K Soong,  “Improving prosody with linguistic and bert derived features in multi-speaker based mandarin chinese neural tts,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6704–6708.
  * [25] Arijit Mukherjee, Shubham Bansal, Sandeepkumar Satpal, and Rupeshkumar Mehta,  “Text aware emotional text-to-speech with bert,”  in Interspeech, 2022.
  * [26] Yihan Wu, Xi Wang, Shaofei Zhang, Lei He, Ruihua Song, and Jian-Yun Nie,  “Self-supervised Context-aware Style Representation for Expressive Speech Synthesis,”  in Proc. Interspeech 2022, 2022, pp. 5503–5507.
  * [27] Minchan Kim, Sung Jun Cheon, Byoung Jin Choi, Jong Jin Kim, and Nam Soo Kim,  “Expressive text-to-speech using style tag,”  in Interspeech, 2021.
  * [28] Dongchao Yang, Songxiang Liu, Rongjie Huang, Guangzhi Lei, Chao Weng, Helen Meng, and Dong Yu,  “Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,”  arXiv preprint arXiv:2301.13662, 2023.
  * [29] Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xuejiao Tan,  “Prompttts: Controllable text-to-speech with text descriptions,”  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2022.
  * [30] Hyun-Wook Yoon, Ohsung Kwon, Hoyeon Lee, Ryuichi Yamamoto, Eunwoo Song, Jae-Min Kim, and Min-Jae Hwang,  “Language model-based emotion prediction methods for emotional speech synthesis systems,”  in Interspeech, 2022.
  * [31] Sarath Sivaprasad, Saiteja Kosgi, and Vineet Gandhi,  “Emotional Prosody Control for Speech Generation,”  in Proc. Interspeech 2021, 2021, pp. 4653–4657.
  * [32] Saiteja Kosgi, Sarath Sivaprasad, Niranjan Pedanekar, Anil Nelakanti, and Vineet Gandhi,  “Empathic machines: using intermediate features as levers to emulate emotions in text-to-speech systems,”  in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022, pp. 336–347.
  * [33] Laurens Van der Maaten and Geoffrey Hinton,  “Visualizing data using t-sne.,”  Journal of machine learning research, vol. 9, no. 11, 2008.
  * [34] Keith Ito and Linda Johnson,  “The lj speech dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [35] Swaraj Dalmia, Kaustav Tamuly, Pulkit Mishra, Shangeth Raaja, Kumarmanas Nethil, and Abhinav Tushar,  “Emotional TTS Dataset in Indian English,” Dec. 2021.
  * [36] Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger,  “Montreal forced aligner: Trainable text-speech alignment using kaldi.,”  in Interspeech, 2017, vol. 2017, pp. 498–502.
  * [37] Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim,  “Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6199–6203.
  * [38] Jaehyeon Kim, Jungil Kong, and Juhee Son,  “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,”  in International Conference on Machine Learning. PMLR, 2021, pp. 5530–5540.
  * [39] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,”  Advances in neural information processing systems, vol. 33, pp. 12449–12460, 2020.


2}}{12}T_{1-\delta}^{\prime}+\Big{(}\frac{1}{2}\theta_{\delta}^{\prime}-\frac{1}{4}\Big{)}h_{1}(1-\delta)+\frac{\sigma^{2}}{2(1-\delta-x)}-\frac{1}{2}\mathcal{J}_{2}^{(\delta)}+\mathcal{O}\Big{(}\frac{M_{1}}{\sqrt{N_{1}}}\Big{)},$
$\displaystyle\frac{1}{N_{1}}\widehat{B}_{7}^{(3)}$
$\displaystyle=\frac{5\sigma^{4}}{24}\frac{\sqrt{N_{1}}}{M_{1}^{3}}+\mathcal{O}\Big{(}\frac{1}{M_{1}^{4}}+\frac{1}{N_{1}^{2}}+\frac{M_{1}}{N_{1}^{2}\sqrt{N_{1}}}\Big{)},$
$\displaystyle\frac{1}{N_{1}^{2}}\widehat{B}_{8}^{(3)}$
$\displaystyle=-\frac{37\sigma^{6}}{120}\frac{\sqrt{N_{1}}}{M_{1}^{5}}+\mathcal{O}\Big{(}\frac{1}{M_{1}^{4}}+\frac{1}{N_{1}^{2}}+\frac{M_{1}}{N_{1}^{2}\sqrt{N_{1}}}\Big{)}.$

Combining these equations, the lemma follows. ∎

###  4.3. Proof of Theorem 1.3

We now complete the proof of Theorem 1.3.

###### Proof of Theorem 1.3.

Since $N_{1}=N+\frac{1}{2}$, Lemma 4.10 gives rise to

$\displaystyle\widehat{S}_{1}$
$\displaystyle=\widehat{B}_{1}N^{2}+\widehat{B}_{2}N\log
N+(\widehat{B}_{1}+\widehat{B}_{3})N+\widehat{B}_{4}\sqrt{N}$
$\displaystyle\quad+\Big{(}\frac{1}{2}\widehat{B}_{2}+\widehat{B}_{5}\Big{)}\log
N+\frac{1}{4}\widehat{B}_{1}+\frac{1}{2}\widehat{B}_{2}+\frac{1}{2}\widehat{B}_{3}+\widehat{B}_{6}+\mathcal{O}\Big{(}\frac{1}{\sqrt{N}}\Big{)}.$

Combining this with Lemma 4.5, we have

$\displaystyle\log\mathcal{P}_{n}^{\mathbb{H}}(R;\alpha,c)$
$\displaystyle=\widehat{A}_{1}^{\prime}N^{2}+\widehat{A}_{2}^{\prime}N\log
N+\widehat{A}_{3}^{\prime}N+\widehat{A}_{4}^{\prime}\sqrt{N}+\widehat{A}_{5}^{\prime}\log
N+\widehat{A}_{6}^{\prime}$
$\displaystyle\quad+\mathcal{O}\Big{(}\frac{1}{M}+\frac{M^{2}}{N}+\frac{M^{4}}{N^{2}}+\frac{\sqrt{N_{1}}}{M_{1}^{7}}+\frac{1}{M_{1}^{2}}+\frac{M_{1}^{5}}{\sqrt{N_{1}}}+\frac{M_{1}^{10}}{N_{1}}\Big{)}$

as $N\to\infty$, where

$\displaystyle\widehat{A}_{1}^{\prime}$
$\displaystyle=\widehat{B}_{1}+\widehat{E}_{1},\qquad\widehat{A}_{2}^{\prime}=\widehat{B}_{2}+\widehat{E}_{2},\qquad\widehat{A}_{3}^{\prime}=\widehat{B}_{1}+\widehat{B}_{3}+\widehat{E}_{3},$
$\displaystyle\widehat{A}_{4}^{\prime}$
$\displaystyle=\widehat{B}_{4},\qquad\widehat{A}_{5}^{\prime}=\frac{1}{2}\widehat{B}_{2}+\widehat{B}_{5}+\widehat{E}_{5},\qquad\widehat{A}_{6}^{\prime}=\frac{1}{4}\widehat{B}_{1}+\frac{1}{2}\widehat{B}_{2}+\frac{1}{2}\widehat{B}_{3}+\widehat{B}_{6}+\widehat{E}_{6}.$

After straightforward simplifications, reusing some computations from the
proof of Theorem 1.1, we obtain

(4.24) $\displaystyle\begin{split}\widehat{A}_{1}^{\prime}&=1-x+\log
x,\qquad\widehat{A}_{2}^{\prime}=-\frac{1}{2}(1-x),\\\
\widehat{A}_{3}^{\prime}&=(1-x)\Big{[}1-\frac{1}{2}\log(4\pi)+\frac{1}{2}\log\Big{(}\frac{x}{1-x}\Big{)}\Big{]}-\Big{(}2c+\frac{1}{2}\Big{)}\log
x,\\\
\widehat{A}_{4}^{\prime}&=\sqrt{\frac{x(1-x)}{2}}\,\mathcal{I}_{1},\qquad\widehat{A}_{5}^{\prime}=-c^{2}+\frac{1}{4}x+\frac{1}{24},\\\
\widehat{A}_{6}^{\prime}&=\Big{(}c^{2}-\frac{1}{4}x-\frac{1}{24}\Big{)}\log\Big{(}\frac{x}{1-x}\Big{)}-\Big{(}c+\frac{1-x}{2}\Big{)}\frac{\log(4\pi)}{2}+\Big{(}c+\frac{1}{4}\Big{)}\log
2\\\
&\quad+\frac{1-2x}{3}\mathcal{I}_{2}+\frac{1}{24}(1+11x)-2\zeta^{\prime}(-1)+\log\Big{(}G(c+1)G(c+\tfrac{3}{2})\Big{)}.\end{split}$

Finally, using the relation $N=n+\alpha+c+\frac{1}{2}$, we can recover the
large $n$ asymptotics of the gap probability:

(4.25)
$\displaystyle\begin{split}\log\mathcal{P}_{n}^{\mathbb{H}}(R;\alpha,c)&=\widehat{A}_{1}^{\prime}n^{2}+\widehat{A}_{2}^{\prime}n\log
n+((2\alpha+2c+1)\widehat{A}_{1}^{\prime}+\widehat{A}_{3}^{\prime})n+\widehat{A}_{4}^{\prime}\sqrt{n}\\\
&\quad+\Big{(}(\alpha+c+\tfrac{1}{2})\widehat{A}_{2}^{\prime}+\widehat{A}_{5}^{\prime}\Big{)}\log
n\\\
&\quad+(\alpha+c+\tfrac{1}{2})^{2}\widehat{A}_{1}^{\prime}+(\alpha+c+\tfrac{1}{2})(\widehat{A}_{2}^{\prime}+\widehat{A}_{3}^{\prime})+\widehat{A}_{6}^{\prime}+\mathcal{O}(n^{-\frac{1}{12}}).\end{split}$

By putting (4.24) to (4.25), the proof of Theorem 1.3 is complete. ∎

##  Appendix A Tail Probabilities of the Binomial Distribution

In this appendix, we compile some tail probabilities of the binomial
distribution.

The following lemma provides an upper bound for the lower tail probability,
which can be found in [96, p.405].

######  Lemma A.1.

Let $X\sim B(n,p)$. Then for any integer $k\leq\frac{n^{2}p}{n+1}-1$, we have

(A.1) $\mathbb{P}(X\geq k)\geq
e^{-np}\sum_{j=k}^{\infty}\frac{(np)^{j}}{j!}\geq
1-\frac{1}{2}\operatorname{erfc}\Big{(}\frac{np-k}{\sqrt{2np}}\Big{)}.$

Let us also introduce a slightly improved version of the asymptotic behaviour
in [59] adapted to our purpose.

######  Lemma A.2.

Let $X\sim B(N,x)$ where $0<x<1$. Let $\delta\in(0,\min\\{\frac{1}{4},1-x\\})$
be a constant. Suppose that $\phi$ is an increasing function such that
$0<\phi(N)<\delta N$ and $\phi(N)\to\infty$ as $N\to\infty$. Then as
$N\to\infty$,

(A.2)
$\displaystyle\begin{split}\log\mathbb{P}[X\geq(1-t)N]&=-\frac{1}{2}\Big{(}t\log\Big{(}\frac{t}{1-x}\Big{)}+(1-t)\log\Big{(}\frac{1-t}{x}\Big{)}\Big{)}N-\frac{1}{2}\log
N-\log\Big{(}\frac{1-x-t}{(1-x)(1-t)}\Big{)}\\\ &\quad-\frac{1}{2}\log(2\pi
t(1-t))+\frac{1}{N}\bigg{(}\frac{1}{12}-\frac{1}{12}\frac{1}{t(1-t)}-\frac{xt}{(1-x-t)^{2}}\bigg{)}+\mathcal{O}\Big{(}\frac{1}{t^{3}N^{3}}+\frac{1}{N^{2}}\Big{)}\end{split}$

uniformly for $t$ such that $tN$ is an integer with $\phi(N)<tN<\delta N$.

###### Proof.

Write

(A.3) $s=1-t,\qquad r=\frac{tx}{(1-t)(1-x)}.$

Then by definition, we have

(A.4)
$\mathbb{P}[X\geq(1-t)N]=\sum_{k=0}^{tN}\binom{N}{tN-k}x^{(1-t)N+k}(1-x)^{tN-k}=\binom{N}{tN}(x^{1-t}(1-x)^{t})^{N}\sum_{k=0}^{tN}r^{k}\prod_{j=1}^{k}\frac{1-\frac{j-1}{tN}}{1+\frac{j}{sN}}.$

By using (2.3), we have

(A.5) $\displaystyle\begin{split}\log\binom{N}{tN}&=-\Big{(}t\log
t+(1-t)\log(1-t)\Big{)}N-\frac{1}{2}\log N\\\ &\quad-\frac{1}{2}\log(2\pi
t(1-t))+1+\frac{1}{12N}\Big{(}1-\frac{1}{t(1-t)}\Big{)}+\mathcal{O}\Big{(}\frac{1}{t^{3}N^{3}}\Big{)}.\end{split}$

Next, we estimate the summation in (A.4). We will compute explicit bounds for
this sum. For this purpose, let

(A.6) $\displaystyle\begin{split}L(k;N,t)&=1-\frac{k^{2}+(2t-1)k}{2stN},\\\
U(k;N,t)&=1-\frac{k^{2}+(2t-1)k}{2stN}+\frac{3k^{4}+(20t-10)k^{3}+(9-24st)k^{2}+(4t-2)k}{24s^{2}t^{2}N^{2}}.\end{split}$

We claim that for any $0\leq k\leq tN$,

(A.7) $L(k;N,t)\leq\prod_{j=1}^{k}\frac{1-\frac{j-1}{tN}}{1+\frac{j}{sN}}\leq
U(k;N,t).$

Fix $N,t$ and let us write $L(k):=L(k;N,t)$, $U(k):=U(k;N,t)$. First we show
the left inequality of (A.7). Since the inequality is trivial if $L(k)\leq 0$,
we may assume $L(k)>0$. Note that it suffices to show

$\frac{L(j)}{L(j-1)}\leq\frac{1-\frac{j-1}{tN}}{1+\frac{j}{sN}}$

for every $j=1,\ldots,k$. Through straightforward computations, we have

$\displaystyle L(j)\Big{(}1+\frac{j}{sN}\Big{)}$
$\displaystyle=1-\frac{j^{2}-j}{2stN}-\frac{j^{3}+(2t-1)j^{2}}{2s^{2}tN^{2}},$
$\displaystyle L(j-1)\Big{(}1-\frac{j-1}{tN}\Big{)}$
$\displaystyle=1-\frac{j^{2}-j}{2stN}+\frac{(j-1)^{2}(j+2t-2)}{2st^{2}N^{2}}.$

It is clear that the numerators of the last terms are non-negative for any
positive integer $j$, thereby proving the lower bound part of (A.7).

Next, let us verify the right inequality of (A.7). Let $0\leq\alpha\leq 1$.
Note that if $tN$ is large enough, we have

$\displaystyle 8s^{2}U(\alpha tN)$
$\displaystyle=t^{2}N^{2}\alpha^{4}+\frac{20t-10}{3}tN\alpha^{3}+(3-8st-4stN)\alpha^{2}+\frac{(2t-1)(2-12stN)}{3tN}\alpha+8s^{2}$
$\displaystyle=\Big{(}tN\alpha^{2}+\frac{10t-5}{3}\alpha-2s\Big{)}^{2}+\Big{(}\frac{2}{9}+\frac{28}{9}st\Big{)}\alpha^{2}+\Big{(}\frac{4t-2}{3tN}+\frac{16}{3}st-\frac{8}{3}s\Big{)}\alpha+4s^{2}$
$\displaystyle\geq-\frac{2}{3tN}-\frac{8}{3}s+4s^{2}\geq\frac{1}{4}-\frac{2}{3tN}\geq
0.$

Therefore, it follows that

(A.8) $U(k)\geq 0$

for all integer $k=0,1,\dots,tN$. Thus it is enough to show again

$\frac{1-\frac{j-1}{tN}}{1+\frac{j}{sN}}\leq\frac{U(j)}{U(j-1)}$

for all $j=1,\dots,tN$. For this, note that

$\displaystyle U(j-1)\Big{(}1-\frac{j-1}{tN}\Big{)}$
$\displaystyle=1-\frac{j^{2}-j}{2stN}+\frac{3j^{4}+(20t-10)j^{3}+(9-24st)j^{2}+(4t-2)j}{24s^{2}t^{2}N^{2}}-\frac{E_{1}(j)}{24s^{2}t^{3}N^{3}},$
$\displaystyle U(j)\Big{(}1+\frac{j}{sN}\Big{)}$
$\displaystyle=1-\frac{j^{2}-j}{2stN}+\frac{3j^{4}+(20t-10)j^{3}+(9-24st)j^{2}+(4t-2)j}{24s^{2}t^{2}N^{2}}+\frac{E_{2}(j)}{24s^{3}t^{2}N^{3}},$

where

$\displaystyle E_{1}(j)$
$\displaystyle=(j-1)^{2}\big{[}(j-2)(j-3)(3j-4)+4t(5(j-1)^{2}-6s(j-1)+1)\big{]},$
$\displaystyle E_{2}(j)$
$\displaystyle=j^{2}[(j-1)(j-2)(3j-1)+4t(5j^{2}-6sj+1)].$

Since the quadratic function $f(u)=5u^{2}-6su+1$ attains its minimum at
$u=\frac{3s}{5}\in(0,1)$ and both $f(0)$ and $f(1)$ are positive, we can
deduce that $f(j)>0$ for all integers $j$. Consequently, we can conclude that
$E_{1}(j)$ and $E_{2}(j)$ are positive for all $j$, which leads to the upper
bound part of (A.7).

Next, we assert that

(A.9)
$\sum_{k=0}^{tN}r^{k}\prod_{j=1}^{k}\frac{1-\frac{j-1}{tN}}{1+\frac{j}{sN}}=\frac{1}{1-r}\Big{(}1-\frac{1}{N}\frac{tx}{(1-x-t)^{2}}+\mathcal{O}\Big{(}\frac{1}{N^{2}}\Big{)}\Big{)}.$

For this, note that for large enough $tN$ and $k>tN$,

(A.10) $\displaystyle\begin{split}L(k;N,t)&=1-\frac{k^{2}+(2t-1)k}{2stN}\leq
1-\frac{(tN)^{2}+(2t-1)tN}{2stN}<0,\\\
U(k;N,t)&=1-\frac{k^{2}+(2t-1)k}{2stN}+\frac{3k^{4}+(20t-10)k^{3}+(9-24st)k^{2}+(4t-2)k}{24s^{2}t^{2}N^{2}}>1\end{split}$

since $3k^{4}/(24s^{2}t^{2}N^{2})$ term dominates the size of $U(k;N,t)$. Let
$N$ be large enough so that $tN>\phi(N)$ satisfies (A.8) and (A.10). Then one
can observe that

$\displaystyle\sum_{k=0}^{\infty}r^{k}L(k;N,t)\leq\sum_{k=0}^{tN}r^{k}L(k;N,t)\leq\sum_{k=0}^{tN}r^{k}\prod_{i=1}^{k}\frac{1-\frac{i-1}{tN}}{1+\frac{i}{sN}}\leq\sum_{k=0}^{\infty}r^{k}U(k;N,t)\leq\sum_{k=0}^{\infty}r^{k}U(k;N,t).$

Here, we have assumed $t<\delta<1-x$ so that $r<1$. Then we obtain

$\displaystyle\sum_{k=0}^{\infty}r^{k}L(t;N,t)$
$\displaystyle=\frac{1}{1-r}\Big{(}1-\frac{1}{N}\frac{tx}{(1-x-t)^{2}}\Big{)},$
$\displaystyle\sum_{k=0}^{\infty}r^{k}U(k;N,t)$
$\displaystyle=\frac{1}{1-r}\Big{(}1-\frac{1}{N}\frac{tx}{(1-x-t)^{2}}+\frac{1}{N^{2}}\frac{tx(1-x^{2}-t(1-2x))}{(1-x-t)^{4}}\Big{)},$

which gives rise to (A.9). Then we conclude the proof by putting the
asymptotics (A.5) and (A.9) to (A.4). ∎

## References

  * [1] K. Adhikari, _Hole probabilities for $\beta$-ensembles and determinantal point processes in the complex plane_, Electron. J. Probab. 23 (2018), 1–21.
  * [2] K. Adhikari and N. K. Reddy, _Hole probabilities for finite and infinite Ginibre ensembles_ , Int. Math. Res. Not. 2017 (2017), 6694–6730.
  * [3] K. Alishahi and M. Zamani, _The spherical ensemble and uniform distribution of points on the sphere_ , Electron. J. Probab. 20 (2015), no. 23, 27 pp.
  * [4] G. Akemann, S.-S. Byun and M. Ebke, _Universality of the number variance in rotational invariant two-dimensional Coulomb gases_ , J. Stat. Phys. 190 (2023), 1–34.
  * [5] G. Akemann, S.-S. Byun, M. Ebke and G. Schehr, _Universality in the number variance and counting statistics of the real and symplectic Ginibre ensemble_ , J. Phys. A 56 (2023), 495202.
  * [6] G. Akemann, M. Ebke and I. Parra, _Skew-orthogonal polynomials in the complex plane and their Bergman-like kernels_ , Comm. Math. Phys. 389 (2022), 621–659.
  * [7] G. Akemann, J. R. Ipsen and E. Strahov, _Permanental processes from products of complex and quaternionic induced Ginibre ensembles_ , Random Matrices Theory Appl. 54 (2014), 1450014.
  * [8] G. Akemann, M. Phillips and L. Shifrin, _Gap probabilities in non-Hermitian random matrix theory_ , J. Math. Phys. 50 (2009), 063504.
  * [9] G. Akemann and E. Strahov, _Hole probabilities and overcrowding estimates for products of complex Gaussian matrices_ , J. Stat. Phys. 151 (2013), 987–1003.
  * [10] Y. Ameur, C. Charlier and J. Cronvall, _Free energy and fluctuations in the random normal matrix model with spectral gaps_ , arXiv:2312.13904.
  * [11] Y. Ameur, C. Charlier, J. Cronvall and J. Lenells, _Disk counting statistics near hard edges of random normal matrices: the multi-component regime_ , Adv. Math. 441 (2024), 109549.
  * [12] Y. Ameur, C. Charlier, J. Cronvall and J. Lenells, _Exponential moments for disk counting statistics at the hard edge of random normal matrices_ , J. Spectr. Theory 13 (2023), 841–902.
  * [13] Y. Ameur, C. Charlier and P. Moreillon, _Eigenvalues of truncated unitary matrices: disk counting statistics_ , Monatsh Math (2023). https://doi.org/10.1007/s00605-023-01920-4, arXiv:2305.08976.
  * [14] J. Baik, R. Buckingham and F. DiFranco, _Asymptotics of Tracy-Widom distributions and the total integral of a Painlevé II function_ , Comm. Math. Phys. 280 (2008), 463–497.
  * [15] F. Balogh, M. Bertola, S.-Y. Lee and K. D. T.-R. McLaughlin, _Strong asymptotics of the orthogonal polynomials with respect to a measure supported on the plane_ , Comm. Pure Appl. Math. 68 (2015), 112–172.
  * [16] F. Balogh, T. Grava and D. Merzi, _Orthogonal polynomials for a class of measures with discrete rotational symmetries in the complex plane_ , Constr. Approx. 46 (2017), 109–169.
  * [17] R. Bauerschmidt, P. Bourgade, M. Nikula and H.-T. Yau, _The two-dimensional Coulomb plasma: quasi-free approximation and central limit theorem_ , Adv. Theor. Math. Phys. 23 (2019), 841–1002.
  * [18] C. Beltrán and A. Hardy, _Energy of the Coulomb gas on the sphere at low temperature_ , Arch. Ration. Mech. Anal. 231 (2019), 2007–2017.
  * [19] G. Ben Arous, A. Dembo and A. Guionnet, _Aging of spherical spin glasses_ , Probab. Theory Related Fields 120 (2001), 1–67.
  * [20] S. Berezin, A. B. J. Kuijlaars and I. Parra, _Planar orthogonal polynomials as type I multiple orthogonal polynomials_ , SIGMA Symmetry Integrability Geom. Methods Appl. 19 (2023), Paper No. 020, 18 pp.
  * [21] M. Bertola, J. G. Elias Rebelo and T. Grava, _Painlevé IV critical asymptotics for orthogonal polynomials in the complex plane_ , SIGMA Symmetry Integrability Geom. Methods Appl. 14 (2018), Paper No. 091, 34pp.
  * [22] E. Blackstone, C. Charlier and J. Lenells, _Oscillatory asymptotics for the airy kernel determinant on two intervals_ , Int. Math. Res. Not. 2022 (2022), 2636–2687.
  * [23] E. Blackstone, C. Charlier and J. Lenells, _Gap probabilities in the bulk of the Airy process_ , Random Matrices Theory Appl. 11 (2022), no.2, Paper No. 2250022, 30 pp.
  * [24] E. Blackstone, C. Charlier and J. Lenells, _The Bessel kernel determinant on large intervals and Birkhoff’s ergodic theorem_ , Comm. Pure Appl. Math. 76 (2023), 3300–3345.
  * [25] R. Butez, D. García-Zelada, A. Nishry and A. Wennman, _Universality for outliers in weakly confined Coulomb-type systems_ , arXiv:2104.03959.
  * [26] S.-S. Byun and P. J. Forrester, _Progress on the study of the Ginibre ensembles I: GinUE_ , arXiv:2211.16223.
  * [27] S.-S. Byun and P. J. Forrester, _Progress on the study of the Ginibre ensembles II: GinOE and GinSE_ , arXiv:2301.05022.
  * [28] S.-S. Byun and P. J. Forrester, _Spherical induced ensembles with symplectic symmetry_ , SIGMA Symmetry Integrability Geom. Methods Appl. 19 (2023), Paper No. 033, 28 pp.
  * [29] S.-S. Byun, N.-G. Kang and S.-M. Seo, _Partition functions of determinantal and Pfaffian Coulomb gases with radially symmetric potentials_ , Comm. Math. Phys. 401 (2023), 1627–1663.
  * [30] S.-S. Byun, S.-M. Seo and M. Yang, _Free energy expansions of a conditional GinUE and large deviations of the smallest eigenvalue of the LUE_ , arXiv:2402.18983.
  * [31] T. Can, P. J. Forrester, G. Téllez and P. Wiegmann, _Exact and asymptotic features of the edge density profile for the one component plasma in two dimensions_ , J. Stat. Phys. 158 (2015), 1147–1180.
  * [32] C. Charlier, _Asymptotics of Muttalib-Borodin determinants with Fisher-Hartwig singularities_ , Selecta Math. 28 (2022), no.3, Paper No. 50, 60 pp.
  * [33] C. Charlier, _Asymptotics of determinants with a rotation-invariant weight and discontinuities along circles_ , Adv. Math. 408 (2022), 108600.
  * [34] C. Charlier, _Large gap asymptotics on annuli in the random normal matrix model_ , Math. Ann. 388 (2024), 3529–3587.
  * [35] C. Charlier, _Hole probabilities and balayage of measures for planar Coulomb gases_ , arXiv:2311.15285.
  * [36] C. Charlier and A. Deaño, _Asymptotics for Hankel determinants associated to a Hermite weight with a varying discontinuity_ , SIGMA Symmetry Integrability Geom. Methods Appl. 14 (2018), Paper No. 018, 43 pp.
  * [37] C. Charlier and R. Gharakhloo, _Asymptotics of Hankel determinants with a Laguerre-type or Jacobi-type potential and Fisher–Hartwig singularities_ Adv. Math. 383 (2021), 107672.
  * [38] C. Charlier and J. Lenells, _Exponential moments for disk counting statistics of random normal matrices in the critical regime_ , Nonlinearity 36 (2023), 1593.
  * [39] C. Charlier, J. Lenells and J. Mauersberger, _Higher order large gap asymptotics at the hard edge for Muttalib-Borodin ensembles_ , Comm. Math. Phys. 384 (2021), 829–907.
  * [40] C. Charlier, J. Lenells and J. Mauersberger, _The multiplicative constant for the Meijer-G kernel determinant_ , Nonlinearity 34 (2021), 2837–2877.
  * [41] T. Claeys, M. Girotti and D. Stivigny, _Large gap asymptotics at the hard edge for product random matrices and Muttalib-Borodin ensembles_ , Int. Math. Res. Not. 2019 (2019), 2800–2847.
  * [42] T. Claeys, I. Krasovsky and O. Minakov, _Weak and strong confinement in the Freud random matrix ensemble and gap probabilities_ , Comm. Math. Phys. 402 (2023), 833–894.
  * [43] J. G. Criado del Rey and A. B. J. Kuijlaars, _A vector equilibrium problem for symmetrically located point charges on a sphere_ , Constr. Approx. 55 (2022), 775–827.
  * [44] F. D. Cunden, F. Mezzadri and P. Vivo, _Large deviations of radial statistics in the two-dimensional one-component plasma_ , J. Stat. Phys. 164 (2016), 1062–1081.
  * [45] D. Dai, S.-X. Xu and L. Zhang, _Asymptotics of Fredholm determinant associated with the Pearcey kernel_ , Comm. Math. Phys. 382 (2021), 1769–1809.
  * [46] D. S. Dean and S. N. Majumdar, _Large deviations of extreme eigenvalues of random matrices_ , Phys. Rev. Lett. 97 (2006), 160201.
  * [47] D. S. Dean and S. N. Majumdar, _Extreme value statistics of eigenvalues of Gaussian random matrices_ , Phys. Rev. E 77 (2008), 041108.
  * [48] P. Deift, A. Its and X. Zhou, _A Riemann-Hilbert approach to asymptotic problems arising in the theory of random matrix models, and also in the theory of integrable statistical mechanics_ , Ann. of Math. 146 (1997), 149–235.
  * [49] P. Deift, A. Its and I. Krasovsky, _Asymptotics of the Airy-Kernel Determinant_ , Comm. Math. Phys. 278 (2008), 643–678.
  * [50] P. Deift, I. Krasovsky and J. Vasilevska, _Asymptotics for a determinant with a confluent hypergeometric kernel_ , Int. Math. Res. Not. 2011 (2011), 2117–2160.
  * [51] A. Deaño and N. Simm, _On the probability of positive-definiteness in the gGUE via semi-classical Laguerre polynomials_ , J. Approx. Theory 220 (2017), 44–59.
  * [52] A. Deaño and N. Simm, _Characteristic polynomials of complex random matrices and Painlevé transcendents_ , Int. Math. Res. Not. 2022 (2022), 210–264.
  * [53] F. Dyson, _Fredholm determinants and inverse scattering problems_ , Comm. Math. Phys. 47 (1976), 171–183.
  * [54] A. Edelman, E. Kostlan and M. Shub, _How many eigenvalues of a random matrix are real?_ J. Amer. Math. Soc. 7 (1994), 247–267.
  * [55] T. Ehrhardt, _Dyson’s constant in the asymptotics of the Fredholm determinant of the sine kernel_ , Comm. Math. Phys. 262 (2006), 317–341.
  * [56] T. Ehrhardt, _The asymptotics of a Bessel-kernel determinant which arises in random matrix theory_ , Adv. Math. 225 (2010), 3088–3133.
  * [57] B. Fahs, _Double scaling limits of Toeplitz, Hankel and Fredholm determinants_ , PhD thesis, Université Catholique de Louvain, 2017.
  * [58] B. Fahs and I. Krasovsky, _Sine-kernel determinant on two large intervals_ , Comm. Pure Appl. Math. 77 (2024), 1958–2029.
  * [59] G. C. Ferrante, _Bounds on binomial tails with applications_ , IEEE Trans. Inform. Theory 67 (2021), 8273–8279.
  * [60] J. Fischmann, W. Bruzda, B.A. Khoruzhenko, H.-J. Sommers and K. Zyczkowski, _Induced Ginibre ensemble of random matrices and quantum operations_ , J. Phys. A 45 (2012), 075203.
  * [61] J. Fischmann and P. J. Forrester, _One-component plasma on a spherical annulus and a random matrix ensemble_ , J. Stat. Mech. Theory Exp. 2011 (2011), P10003.
  * [62] P. J. Forrester, _Some statistical properties of the eigenvalues of complex random matrices_ , Phys. Lett. A 169 (1992), 21–24.
  * [63] P. J. Forrester, _Log-gases and random matrices_ , Princeton University Press, Princeton, NJ, 2010.
  * [64] P. J. Forrester and M. Krishnapur, _Derivation of an eigenvalue probability density function relating to the Poincaré disk_ , J. Phys. A, 42 (2009), 385204.
  * [65] P. J. Forrester and A. Mays, _Pfaffian point process for the Gaussian real generalised eigenvalue problem_ , Probab. Theory Relat. Fields 154 (2012), 1–47.
  * [66] P. J. Forrester and N. S. Witte, _Asymptotic forms for hard and soft edge general $\beta$ conditional gap probabilities_, Nuclear Phys. B 859, (2012), 321–340.
  * [67] Y. V. Fyodorov and B. A. Khoruzhenko, _Systematic analytical approach to correlation functions of resonances in quantum chaotic scattering_ , Phys. Rev. Lett. 83 (1999), 65–68.
  * [68] S. Ghosh and A. Nishry, _Point processes, hole events, and large deviations: random complex zeros and Coulomb gases_ , Constr. Approx. 48 (2018), 101–136.
  * [69] S. Ghosh and A. Nishry, _Gaussian complex zeros on the hole event: the emergence of a forbidden region_ , Comm. Pure Appl. Math. 72 (2019), 3–62.
  * [70] R. Grobe, F. Haake and H.-J. Sommers, _Quantum distinction of regular and chaotic dissipative motion_ , Phys. Rev. Lett. 61 (1998), 1899–1902.
  * [71] B. Jancovici, J. Lebowitz and G. Manificat, _Large charge fluctuations in classical Coulomb systems_ , J. Stat. Phys. 72 (1993), 773–787.
  * [72] B. Jancovici, G. Manificat and C. Pisani, _Coulomb systems seen as critical systems: finite-size effects in two dimensions_ , J. Stat. Phys. 76 (2019), 307–329.
  * [73] E. Katzav and I. P. Castillo, _Large deviations of the smallest eigenvalue of the Wishart–Laguerre ensemble_ , Phys. Rev. E 82 (2010), 040104.
  * [74] S. Klevtsov, _Random normal matrices, Bergman kernel and projective embeddings_ , J. High Energy Phys. 133 (2014), no. 1, 18 pp.
  * [75] I. Krasovsky, _Gap probability in the spectrum of random matrices and asymptotics of polynomials orthogonal on an arc of the unit circle_ , Int. Math. Res. Not. 2004 (2004), 1249–1272.
  * [76] I. Krasovsky and T.-H. Maroudas, _Airy-kernel determinant on two large intervals_ , Adv. Math. 440 (2024), Paper No. 109505, 79 pp.
  * [77] M. Krishnapur, _From random matrices to random analytic functions_ , Ann. Probab. 37 (2009), 314–346.
  * [78] T. Krüger, S.-Y. Lee and M. Yang, _Local statistics in normal matrix models with merging singularity_ , arXiv:2306.12263.
  * [79] T. Leblé and S. Serfaty, _Large deviation principle for empirical fields of log and Riesz gases_ , Invent. Math. 210 (2017), 645–757.
  * [80] S.-Y. Lee and M. Yang, _Discontinuity in the asymptotic behavior of planar orthogonal polynomials under a perturbation of the Gaussian weight_ , Comm. Math. Phys. 355 (2017), 303–338.
  * [81] S.-Y. Lee and M. Yang, _Planar orthogonal polynomials as Type II multiple orthogonal polynomials_ , J. Phys. A 52 (2019), 275202.
  * [82] S.-Y. Lee and M. Yang, _Strong asymptotics of planar orthogonal polynomials: Gaussian weight perturbed by finite number of point charges_ , Comm. Pure Appl. Math. 76 (2023), 2888–2956.
  * [83] A. Legg and P. Dragnev, _Logarithmic equilibrium on the sphere in the presence of multiple point charges_ , Constr. Approx. 54 (2021), 237–257.
  * [84] S. N. Majumdar and G. Schehr, _Top eigenvalue of a random matrix: Large deviations and third order phase transition_ , J. Stat. Mech. 2014 (2014), P01012.
  * [85] S. N. Majumdar and M. Vergassola, _Large deviations of the maximum eigenvalue for Wishart and Gaussian random matrices_ , Phys. Rev. Lett. 102 (2009), 060601.
  * [86] O. Marchal, _Asymptotic expansion of Toeplitz determinants of an indicator function with discrete rotational symmetry and powers of random unitary matrices_ , Lett. Math. Phys. 113 (2023), 78.
  * [87] A. Mays, _A real quaternion spherical ensemble of random matrices_ , J. Stat. Phys. 153 (2013), 48–69.
  * [88] A. Mays and A. Ponsaing, _An induced real quaternion spherical ensemble of random matrices_ , Random Matrices Theory Appl. 6 (2017), 1750001.
  * [89] P. Moreillon, in preparation.
  * [90] A. Nishry and A, Wennman, _The forbidden region for random zeros: appearance of quadrature domains_ , Comm. Pure Appl. Math. 77 (2024), 1766–1849.
  * [91] K. Noda, _Determinantal structure of the overlaps for induced spherical unitary ensemble_ , arXiv:2312.12690.
  * [92] F. W. J. Olver, D. W. Lozier, R. F. Boisvert and C. W. Clark., eds. _NIST Handbook of Mathematical Functions_ , Cambridge: Cambridge University Press, 2010.
  * [93] L. Samaj, _Finite-size effects in non-neutral two-dimensional Coulomb fluids_ , J. Stat. Phys. 168 (2017), 434–446.
  * [94] E. B. Saff and V. Totik, _Logarithmic Potentials with External Fields_ , Grundlehren der Mathematischen Wissenschaften, Springer-Verlag, Berlin, 1997.
  * [95] S. Serfaty, _Lectures on Coulomb and Riesz Gases_ , 2024.
  * [96] E. V. Slud, _Distribution inequalities for the binomial law_ , Ann. Probab. 5 (1997), 404–412.
  * [97] G. Téllez and P. J. Forrester, _Exact finite-size study of the 2D OCP at $\Gamma=4$ and $\Gamma=6$_, J. Stat. Phys. 97 (1999), 489–521.
  * [98] N. M. Temme, _Special functions: An introduction to the classical functions of mathematical physics_ , John Wiley & Sons (1996).
  * [99] C. Tracy and H. Widom, _Level-spacing distributions and the Airy kernel_ , Comm. Math. Phys. 159 (1994), 151–174.
  * [100] C. Tracy and H. Widom, _Level spacing distributions and the Bessel kernel_ , Comm. Math. Phys. 161 (1994), 289–309.
  * [101] P. Vivo, S. N. Majumdar and O. Bohigas, _Large deviations of the maximum eigenvalue in Wishart random matrices_ , J. Phys. A 40 (2007), 4317–4337.
  * [102] C. Webb and M. D. Wong, _On the moments of the characteristic polynomial of a Ginibre random matrix_ , Proc. Lond. Math. Soc. 118 (2019), 1017–1056.
  * [103] H. Widom, _The strong Szegö limit theorem for circular arcs_ , Indiana Univ. Math. J. 21 (1971), 277–283.
  * [104] H. Widom, _Asymptotics for the Fredholm determinant of the sine kernel on a union of intervals_ , Comm. Math. Phys. 171 (1995), 159–180.
  * [105] L. Yao and L. Zhang, _Asymptotics of the hard edge Pearcey determinant_ , SIAM J. Math. Anal. 56 (2024), 137–172.
  * [106] L. Yao and L. Zhang, _On the gap probability of the tacnode process_ , Adv. Math. 438 (2024), Paper No. 109474, 85 pp.
  * [107] A. Zabrodin and P. Wiegmann, _Large- $N$ expansion for the 2D Dyson gas_, J. Phys. A 39 (2006), 8933–8964.
  * [108] K. Zyczkowski and H.-J. Sommers, _Truncations of random unitary matrices_ , J. Phys. A 33 (2000), 2045–2057.


Stars and their variability observed from space

# The prototype star $\gamma$ Doradus observed by TESS

S. Christophe  LESIA, Observatoire de Paris, Université PSL, CNRS, Sorbonne
Université, Université de Paris, 5 place Jules Janssen, 92195 Meudon, France
V. Antoci  Stellar Astrophysics Centre, Department of Physics and Astronomy,
Aarhus University, Ny Munkegade 120, DK-8000 Aarhus C, Denmark  E. Brunsden
Department of Physics, University of York, Heslington, York, YO10 5DD, UK
R.-M. Ouazzani1 S.J.A.J. Salmon  STAR Institute, Université de Liège, Allée du
6 Août 19, 4000 Liège, Belgium

###### Abstract

$\gamma$ Doradus is the prototype star for the eponymous class of pulsating
stars that consists of late A-early F main-sequence stars oscillating in low-
frequency gravito-inertial modes. Being among the brightest stars of its kind
(V = 4.2), $\gamma$ Dor benefits from a large set of observational data that
has been recently completed by high-quality space photometry from the TESS
mission. With these new data, we propose to study $\gamma$ Dor as an example
of possibilities offered by synergies between multi-technical ground and
space-based observations. Here, we present the preliminary results of our
investigations.

###### keywords:

asteroseismology – stars: oscillations – stars: rotation – stars: individual:
gamma Doradus

##  1 TESS photometry

TESS observed $\gamma$ Dor during Sectors 3, 4, and 5, representing 80 days of
nearly uninterrupted data. Bright as it is, the star saturates the CCDs and
leaves bleeding trails along CCD columns. This is shown on the Full Frame
Image (FFI) cutout on the left panel of Fig. 1. The lightcurve was extracted
by performing simple aperture photometry on FFI cutouts with a custom mask
chosen to include most of the target flux. Data downlink or scattered light
from the Earth or the Moon are responsible for gaps in the data but overall, a
duty cycle of 87% is achevied.

Figure 1: Left: Example of TESS Full Frame Image (FFI) cutout for $\gamma$
Dor. Right: Lightcurve of $\gamma$ Dor extracted from TESS FFI cutouts.

Figure 2: Amplitude periodograms before (top) and after pre-whintening
(bottom) of the first two frequencies of high amplitude. Red circles represent
frequencies extracted from the TESS lightcurve. Upside down triangles indicate
frequencies found by Brunsden et al. (2018) from ground-based photometry and
spectroscopy.

##  2 Asteroseismology from ground and space

We carried out the frequency analysis of the TESS lightcurve by iterative
prewhitening. We found 21 frequencies with S/N $>4$ (see Fig. 2). The 6
previously known frequencies from ground-based observations (Brunsden et al.
2018) are all confirmed with TESS data. Based on line profile variations,
Brunsden et al. (2018) identified the three frequencies at 15.3, 15.8 and 21.7
$\mu$Hz to be prograde gravity modes of $(\ell,m)=(1,-1)$. Assuming the
frequency group at $\sim$16 $\mu$Hz is mostly consisting of $(1,-1)$ modes, we
applied the method of Christophe et al. (2018) to estimate the near-core
rotation rate of $\gamma$ Dor. The mode frequencies are compatible with
rotation rates in the 9-12 $\mu$Hz range but the limited resolution of the
periodogram prevent us from obtaining a more precise value. The second
frequency group around $\sim$32 $\mu$Hz can be interpreted as either
combination frequencies or $(\ell,m)=(2,-2)$ modes.

##  3 Surface rotation

In order to determine the surface rotation rate, we have need of the
luminosity $L$, the effective temperature $T_{\rm eff}$, the projected surface
velocity $v\sin i$ and the inclination angle $i$. The luminosity is adopted
from Gaia DR2 ($L=6.99\pm 0.06~{}L_{\odot}$, Gaia Collaboration et al. 2018).
We derive $T_{\rm eff}\approx 7145\pm 150$ K by averaging the photometric and
spectroscopic estimates available in the literature. The projected velocity
$v\sin i=59.5\pm 3~{}{\mathrm{km~{}s^{-1}}}$ is taken from Ammler-von Eiff &
Reiners (2012). $\gamma$ Dor hosts a debris disc that has been observed with
Herschel and modelled by Broekhoven-Fiene et al. (2013), who found $i\approx
70^{\circ}$. Assuming the rotation axis of $\gamma$ Dor is aligned with its
disc axis, the surface rotation rate is estimated to be $8.4\pm 0.8$ $\mu$Hz.
This is close to the range of near-core rotation rates estimated from
seismology suggesting a nearly uniform rotation profile.

###### Acknowledgements.

This research made use of Lightkurve, a Python package for Kepler and TESS
data analysis (Lightkurve Collaboration et al. 2018); Period04 (Lenz & Breger
2005) and Astroquery (Ginsburg et al. 2019). S.C. acknowledges support from
the Programme National de Physique Stellaire (PNPS) of the CNRS/INSU co-funded
by CEA and CNES.

## References

  * Ammler-von Eiff & Reiners (2012) Ammler-von Eiff, M. & Reiners, A. 2012, A&A, 542, A116
  * Broekhoven-Fiene et al. (2013) Broekhoven-Fiene, H., Matthews, B. C., Kennedy, G. M., et al. 2013, ApJ, 762, 52
  * Brunsden et al. (2018) Brunsden, E., Pollard, K. R., Wright, D. J., De Cat, P., & Cottrell, P. L. 2018, MNRAS, 475, 3813
  * Christophe et al. (2018) Christophe, S., Ballot, J., Ouazzani, R. M., Antoci, V., & Salmon, S. J. A. J. 2018, A&A, 618, A47
  * Gaia Collaboration et al. (2018) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2018, A&A, 616, A1
  * Ginsburg et al. (2019) Ginsburg, A., Sipőcz, B. M., Brasseur, C. E., et al. 2019, AJ, 157, 98
  * Lenz & Breger (2005) Lenz, P. & Breger, M. 2005, Communications in Asteroseismology, 146, 53
  * Lightkurve Collaboration et al. (2018) Lightkurve Collaboration, Cardoso, J. V. d. M., Hedges, C., et al. 2018, Lightkurve: Kepler and TESS time series analysis in Python, Astrophysics Source Code Library