[affiliation=1]Shareef BabuKalluri [affiliation=1]PrachiSingh
[affiliation=2]Pratik Roy Chowdhuri [affiliation=1]ApoorvaKulkarni
[affiliation=2]Shikha Baghel [affiliation=3]Pradyoth Hegde
[affiliation=3]Swapnil Sontakke [affiliation=3]Deepak K T [affiliation=4]S. R.
Mahadeva Prasanna [affiliation=2]Deepu Vijayasenan [affiliation=1]Sriram
Ganapathy

# The Second DISPLACE Challenge : DIarization of SPeaker and LAnguage in
Conversational Environments

###### Abstract

The DIarization of SPeaker and LAnguage in Conversational Environments
(DISPLACE) 2024 challenge is the second in the series of DISPLACE challenges,
which involves tasks of speaker diarization (SD) and language diarization (LD)
on a challenging multilingual conversational speech dataset. In the DISPLACE
2024 challenge, we also introduced the task of automatic speech recognition
(ASR) on this dataset. The dataset containing $158$ hours of speech,
consisting of both supervised and unsupervised mono-channel far-field
recordings, was released for LD and SD tracks. Further, $12$ hours of close-
field mono-channel recordings were provided for the ASR track conducted on $5$
Indian languages. The details of the dataset, baseline systems and the leader
board results are highlighted in this paper. We have also compared our
baseline models and the team's performances on evaluation data of
DISPLACE-2023 to emphasize the advancements made in this second version of the
challenge.

###### keywords:

Speaker diarization, language diarization, ASR, code-mixing, conversational
speech, DISPLACE challenge.

##  1 Introduction

††This work was funded by the project National Language Translation Mission
(NLTM): BHASHINI, the Ministry of Electronics and Information Technology
(MeitY), Government of India SP/MITO-22-001 grant, and the British Telecom
(BT) grants on conversational speech analytics.

In multilingual cultures, social interactions frequently comprise code-mixed
or code-switched speech [1, 2]. Code-mixing occurs when morphemes or words
from a secondary language are utilized in a primary language phrase (Eg.:
``Wirst du mitmachen, um mit mir ein IPL-Match anzusehen?'' (Will you join to
watch IPL match with me ?)). In contrast, code-switching involves modifying
the conversational language itself at the sentence or phrase level. (Eg.: ``
I'm busy today but, Ich kann beim nächsten Spiel mitmachen'' (I'm busy today
but, I can join for next match)). In multilingual communities like Asia,
Europe, America and some parts of the African continent, code-mixing and code-
switching are more frequent in social conversations [3, 4, 5].

The code-mixed or code-switched instances pose significant challenges for
speech-based systems, such as speaker and language identification or automatic
speech recognition (ASR). In multi-speaker and multilingual scenarios, the
task of identifying ``who spoke when" and ``which language was spoken when",
termed as speaker diarization (SD) and language diarization (LD) respectively,
are significantly challenging. We find that the current speech processing
systems are ill-equipped to perform these tasks meaningfully [6].

In this paper, we detail our efforts in extending the first DISPLACE challenge
conducted in $2023$ [6]. We created a dataset for the second DISPLACE
challenge111https://displace2024.github.io/ from four different academic
institutes preserving the same recording settings for the entire dataset. The
data reflects the social interactions in multilingual communities with code-
mixed or code-switched speech, natural overlaps, reverberation, and noise. The
key highlights of the DISPLACE-2024 challenge are,

  * •

Introducing the track on speech recognition which attempts to investigate
speech transcription in code-mixed multi-speaker settings on $5$ different
languages.

  * •

Releasing $38$ hours of annotated data and $120$ hours of unsupervised data.

  * •

Updated baseline systems on speaker and language diarization, which improved
the benchmark significantly over the DISPLACE-2023 challenge.

  * •

A leader-board platform 222https://codalab.lisn.upsaclay.fr/competitions/17682
for all $3$ tracks for participants to monitor their progress in system
development.

Figure 1: The DISPLACE 2024 Challenge Timeline.

##  2 Related work

###  2.1 Speaker Diarization

Early research in speaker diarization was predominantly influenced by the
evaluations conducted by the National Institute of Standards and Technology-
Rich Transcription (NIST-RT) [7] on broadcast news (BN) and informal telephone
conversations in English. The Diarization Error Rate (DER), which continues to
be the primary evaluation metric for SD systems, was also proposed during the
NIST-RT evaluations. A series of SD works by Fiscus et al. [8] used in-domain
conversations from meetings. Recently, there have been several evaluation
challenges on SD, namely, the DIHARD challenge [9], VoxSRC-20 [10] using
YouTube videos and Fearless Steps Series [11] in multi-party and multi-stream
naturalistic audio.

###  2.2 Language Diarization

For the past decade, language diarization has been one of the notable research
domains in the multilingual speech-processing research community [12]. Works
carried out for LD tasks mainly used broadcast datasets [2], and recorded in
closed environments [13]. Recently, MERLIon CCS Challenge on language
identification and diarization used code-switched child-directed speech [14].

###  2.3 Automatic Speech Recognition

ASR systems are majorly developed for monolingual scenarios in clean speech
data and are less efficient on low-resourced speech data [15, 16]. ASR
Challenges at Interspeech $2018$ have targeted read-out and conversational
speech data in low-resource Indian languages like Tamil, Telugu, and Gujarati
[16] and scripted telephone conversations of Assamese, Bengali, Tamil, Urdu,
and Hindi [17]. A series of ASR challenges [18, 19], on Tamil, Hindi, and
Indian-English targeted lecture data (formal style) and read speech. But code-
switching scenarios were explored in limited languages like Bengali-English,
Hindi-English [20]. There have been efforts in non-Indian languages, such as
English and Arabic, to tackle speech recognition in code-switching [21] and
the Oriental Language Recognition challenge for multilingual societies [22].

Despite several recent efforts to provide real labeled conversational datasets
through various challenges [9, 11, 10, 14], there is still a need for a more
generic dataset that can capture multi-speaker, multilingual, code-switched
scenarios that are common in day-to-day conversations. DISPLACE 2024 challenge
tries to address this need by providing a natural multilingual conversational
dataset without any restrictions on speakers, languages, topics, etc. The
challenge evaluates the SD, LD, and ASR performance on the same dataset to
make it more meaningful.

##  3 DISPLACE Corpus Details

DISPLACE corpus has been collected from four different academic institutions.
Each institute followed the same recording protocol and instrument
specifications but differed in the recording room settings like size, shape,
and acoustic properties. The recorded conversations contained different
speakers across different age groups and different regions of India. The
procedure in participant selection ensured their ability to converse fluently
with each other in one of the Indian languages (L1) along with Indian-English.
Each recording session lasted about $30$-$60$ minutes, with $3$-$5$
participants conversing in their native language, along with natural code-
mixing and code-switching instances of Indian English. The topics for the
conversations were mainly about culture, lifestyles, entertainment, and
sports, excluding emotionally sensitive or personal topics. The data is
collected using lapel microphones and omnidirectional microphone recorders for
close-field and far-field recordings respectively. More details of the
recording setup and participant selection are given in [6, 23].

Professional annotators were hired to label the data using close-field
recordings for all the targeted speakers in a session. This annotation process
involves marking the details of the speaker, language, and transcript of
speech region. A single Rich Transcription Time Marked (RTTM) annotation file
is generated for each conversation by combining the annotations obtained from
all the participants' lapel microphones. Multiple quality checks were
performed on the annotations before we released them to the participants. More
details on the data annotation and labeling can be found in [23].

###  3.1 Development and Evaluation set

In DISPLACE 2024 challenge, we have released development and evaluation data
to the participants through Zenodo (a cloud service) with a password-protected
link333https://zenodo.org/records/10669296. As was done for the DISPLACE-2023
challenge, no training data was provided to the participants, and they were
allowed to use any public data resources and /or proprietary data to train
their systems.

In this challenge, we have released both supervised (labeled) and unsupervised
(unlabeled) data containing $666$ unique speakers, out of which $459$ are male
and $207$ are female speakers. All the speakers fall in the age group of
$17$-$65$ years. There are $9$ different Indian languages spoken in the $237$
sessions, along with Indian English. For the second DISPLACE challenge, we
have combined the previous challenge's annotated data with $20$% of additional
annotated data. In total, we have annotated data of $38$ hours of
conversational speech from $67$ sessions with $197$ speakers. The data is pre-
processed for volume normalization, all the close-field recordings are time-
aligned with the far-field recordings, and the audio is resampled to $16$ kHz
and normalized to the [-1,1] range.

Development set: We released far-field supervised (labeled) and unsupervised
(unlabeled) conversational data of around $140$ hours as a part of the
development (Dev) set to all the registered participants for SD and LD tracks.
For the ASR task, we released $4$ hours of supervised data for both far-field
and close-field recordings. We have released annotated data of $20$ hours of
conversational speech from $35$ sessions with $98$ speakers as a part of
supervised Dev data for diarization tracks. The sessions chosen for speaker
and language diarization tracks are identical, and the labels are provided
separately for each track in RTTM format. We have released supervised data for
the ASR track for four Indian languages (i.e., Bengali, Hindi, Kannada,
Telugu) and Indian-accented English. Each native language has one hour of
labeled transcripts for close and far-field recordings. For the ASR track, we
provided the segment files to identify the speech regions along with the
transcript labels in native language script.

Unsupervised Data: We released unsupervised data in the second DISPLACE
challenge. In total, we distributed more than $120$ hours of conversational
speech data from $170$ sessions with $493$ speakers from Indian languages like
Hindi, Bengali, Kannada, Telugu, Malayalam, Tamil, Marathi, Assamese, Odiya
and Indian English. This unsupervised data was meant to be useful for adapting
the models to the environmental conditions of the DISPLACE evaluation data for
all three tracks.

Evaluation set: As a part of the evaluation (Eval) set, we released the
supervised conversational data of $18$ hours from $99$ speakers in $32$
sessions. We use the same far-field evaluation data for SD and LD tasks. For
the ASR task, we have released $8$ hours of the close-field recordings.

##  4 Challenge Tasks and Organisation

Participants were encouraged to build their own speech activity detection
systems. The submissions are evaluated based on speech regions ignoring non-
speech regions like background speech, noise, laughing, or clapping. The
second DISPLACE challenge had the following tracks,

  * •

Track-1: Speaker Diarization in multilingual scenarios.

  * •

Track-2: Language Diarization in multi-speaker settings.

  * •

Track-3: ASR on single-speaker code-mixed settings.

The metric for evaluating the system performance of diarization systems is the
diarization error rate (DER) while word error rate (WER) is used for
evaluating speech recognition systems [24]. The challenge's timeline is shown
in Fig 1.

##  5  Baseline Systems

###  5.1 Speaker Diarization

The speaker diarization system follows the steps outlined in the DISPLACE 2023
challenge baseline [6], which includes speech activity detection, segmenting
audio into $1.5$s chunks with a $250$ms shift, followed by extracting
x-vectors using a pre-trained $13$-layer ETDNN model [25]. These x-vectors are
utilized to generate PLDA similarity scores for spectral clustering. The
number of speakers is decided based on the threshold set according to the best
DER on the dev set. We perform Variational Bayes (VB) re-segmentation [26]
using the clustering results to generate the final output. Compared to the
previous year's baseline, we have integrated Pyannote speech activity
detection [27] and overlap detection modules. Based on the posterior
probabilities from the VB-hidden Markov model (HMM) module and the output of
the overlap detection system, we predict up to two speakers for each audio
segment.

###  5.2 Language Diarization

The baseline system follows a two-stage methodology, comprising feature
extraction and clustering. We used the Pyannote speech activity detection [27]
for detecting the speech regions. We then segment the speech regions into
short overlapping segments of $10$s with a $200$ms shift. Unlike DISPLACE 2023
baseline, which relied on language identification (LID) ECAPA-TDNN model
embeddings [6], we employ a deep multitask model called Whisper [28] for the
feature extraction. The Whisper model is trained on the Voxlingua107 dataset
[29], containing data from $99$ languages and $6628$ hours. Notably, the LID
system is trained on $30$s speech segments. Upon feature extraction, the
Whisper language detector [30] furnishes $99$ language posterior
probabilities. These extracted posterior features serve as input for an
agglomerative clustering algorithm. We use a cosine similarity-based distance
metric computed using the posteriors from the Whisper model for clustering.
Subsequently, we enhance our results by employing a Variational Bayes x-vector
(VBx) model [31] for further refinement.

We tested our baseline system with DISPLACE-2023 data and report the
development (dev) and evaluation (Eval) DER for both the models (DISPLACE-2023
and the DISPLACE-2024 baseline system) in Table 1. We observe absolute
improvement of $5.94$% on dev and $12.11$% on the Eval data, respectively, in
terms of DER for the LD task, whereas it is more than 2% absolute improvement
in DER for SD task on both dev and Eval sets.

Table 1: DER ($\%$) comparison of LD and SD baseline systems using DISPLACE 2023 and DISPLACE 2024 challenge models on DISPLACE 2023 data.  | LD | SD
---|---|---
Baseline | Dev | Eval | Dev | Eval
DISPLACE-2023 [23] | 46.95 | 41.67 | 27.33 | 32.18
DISPLACE-2024 | 41.01 | 29.56 | 25.45 | 29.96

###  5.3 Automatic Speech Recognition

As part of the dev and eval phases, we have released Hindi, Bengali, Kannada,
and Telugu conversations. We have provided the segment boundaries and ground
truth transcripts for the dev data, while only segment boundary labels were
released for the eval data. As a part of the baseline system for ASR track, we
have implemented the Google Speech-to-Text cloud services using the close
field recordings of development data [32]. To compute the WER, we use Sclite
toolkit [24], and the WER for dev and eval sets are tabulated in Table 2. As
the data was significantly challenging, we observe that the baseline WER is
similar to those observed in other ASR challenges like CHiME-6 multi-speaker
far-field conversational dataset [33]. Furthermore, the DISPLACE data had
code-switched speech, making the ASR task even more demanding.

Table 2: Comparison of Track 3- ASR baseline WER with top performing team  | Baseline | T1
---|---|---
Language | Dev | Eval | Dev | Eval
Bengali | 63.5 | 75.23 | 54.79 | 63.87
Hindi | 58.5 | 60.43 | 38.90 | 52.27
Kannada | 80.8 | 81.08 | 65.14 | 62.88
Telugu | 71.2 | 66.92 | 59.35 | 57.87
Indian-English | 66.5 | 66.76 | 25.53 | 34.56
Overall | 66.7 | 67.78 | 37.30 | 47.34
Figure 2: DER ($\%$) comparison of best of Track 1 Eval Phases of Teams submissions with baseline. Figure 3: DER ($\%$) comparison of best of Track 2 Eval Phases of Teams submissions with baseline. Table 3: Comparison of top 3 teams model submissions for SD and LD tracks in terms of DER ($\%$), DISPLACE-2024 vs DISPLACE 2023 models using DISPLACE-2023 challenge Eval data. | Track-1:SD | Track-2:LD
---|---|---
Team No. | 2024 | 2023 | 2024 | 2023
T1 | 21.27 | 27.8 | 25.05 | 37.6
T2 | 24.61 | 28.6 | 28.57 | 40.2
T3 | 24.43 | 31.5 | 29.83 | 41.2

##  6 Summary of Challenge Results

###  6.1 Track 1: Speaker Diarization

A total of $11$ teams participated in Eval-Phase 1 & Eval-Phase 2 and eight
teams outperformed the baseline. Figure 2 shows the DER distribution across
the teams. We observe a significant boost in the DER performance in
DISPLACE-2024 compared to the best DER obtained in DISPLACE-2023 challenge
(absolute improvement of $6.53$% in DER reported in Table 3).
Top performing systems: The best single system in T1 submission is the Wav-LM-
based speaker segmentation model from pyannote.audio [27] fine-tuned on the
DISPLACE-2024 Dev set. The model comprises of end-to-end diarization step
followed by global clustering, where the number of speakers is restricted to
$7$. The final submission involved an ensemble of Pyannote based models with
different configurations, permutation invariant training (PIT) for speaker
diarization and mixture invariant training (MixIT) for speech separation
(PixIT) [34] combined using DOVER-Lap. Team T2 experimented with different
pre-trained embedding extractors of the ResNet series [35], and performed
spectral clustering and VBx to generate the output. ResNet-152 with spectral
clustering achieves the best single system DER. Finally, the submission
contained a fusion of the top $7$ models followed by Pyannote overlap
detection [27]. Team T3 focused on improving voice activity detection (VAD)
and overlap detection components. For VAD, the team explored MarbleNet [36]
and Pyannote SAD [27] models trained using the supervised and unsupervised dev
data. The best single system comprises multiscale Titanet-L embedding
extractor followed by spectral clustering.

In Table 3, we compare the top three teams (T1, T2, and T3) of DISPLACE-2024
on the same Eval data of DISPLACE-2023. This table highlights significant
relative improvements for SD with an average relative improvement of $23.49$%
in DER for the top-performing teams.

###  6.2 Track 2: Language Diarization

There were submissions from $6$ teams out of which $3$ teams outperformed over
the baseline. The DER distribution across all teams is shown in Figure 3.

Top performing systems: T1 system used a wav2vec-BERT model [37] to extract
language embeddings from 5s segments with 1s shift. T1 trained a probabilistic
linear discriminant analysis (PLDA) model using the DISPLACE ASR dev data. The
model is fine-tuned on NIST LRE and SRE data and the dev set. This is followed
by PLDA similarity scoring, AHC and VBx re-segmentation to generate the final
output. T2 system used ResNet34 [38]and wav2vec 2.0 architectures for
extracting the language embeddings, utilized spectral clustering of language
embeddings within a sliding window and a heuristic bypass (HBP) method for
similarity matrix computation. T3 system used Pyannote [27] for VAD and
features were extracted using the pre-trained Whisper model [28]. Finally,
agglomerative hierarchical clustering was used to cluster and assign language
cluster labels to the segments.

In Table 3, we compare the top three teams (T1, T2, and T3) on the same eval
data subset of DISPLACE-2023. We observe the relative improvement in the model
for LD is around $33.37$% DER for the top-performing team.

###  6.3 Track 3: ASR

For Track 3, we have two valid submissions from the participants. The top-
performing team (T1) fine-tuned their model using Whisper [28] for English,
Hindi, and Bengali languages. The Bhashini model [39] was deployed for Kannada
and Telugu inferences. This system showed an absolute improvement of $20.4$%
in WER over the baseline system for close-field recordings. The individual
monolingual WER performance along with overall system performance is given
Table 2.

##  7 Summary

This paper provides a comprehensive overview of the second DISPLACE challenge,
which aims to encourage research in processing multi-lingual multi-speaker
conversational audio. This challenge contained three tracks: 1) speaker
diarization, 2) language diarization and 3) speech recognition. Track-1 and
Track-2 share the common data, while Track-3 contained $12$ hours of audio
data in five different Indian languages. As part of the challenge, we released
updated baseline systems for the SD and LD tracks which provided improved
performance over the first DISPLACE challenge benchmarks. The wide
participation for the second DISPLACE challenge across the globe, resulted in
significant improvements over the baseline system for both the SD and LD
tracks. The ASR track using close-field recordings was observed to be
significantly challenging and thus resulted in limited participation. The
best-performing system achieved a WER of $47$% on the evaluation data,
highlighting the need for continued efforts on this demanding dataset. We also
compared our baseline models and top performing teams with the corresponding
results from the DISPLACE-2023 challenge. This comparison led to understand
the progress made in processing multilingual, multi-speaker conversational
data under the DISPLACE challenge series.

##  8 Acknowledgements

The authors would like to thank Kaustubh Kumar and Lokesh Kumar from IIT
Dharwad for helping with data collection. Also Michael free and Shakti
Srivastava from BT for valuable discussions.

## References

  * [1] B. E. Bullock and A. J. E. Toribio, _The Cambridge handbook of linguistic code-switching._ Cambridge University Press, 2009.
  * [2] E. Yilmaz, M. McLaren, H. van den Heuvel, and D. A. van Leeuwen, ``Language diarization for semi-supervised bilingual acoustic model training,'' in _Proc. IEEE ASRU_ , 2017, pp. 91–96.
  * [3] P. Auer, _Code-switching in conversation: Language, Interaction and Identity_. Routledge, 2013.
  * [4] K. Potowski and J. Rothman, _Bilingual youth: Spanish in English-speaking societies_. John Benjamins Publishing, 2011, vol. 42.
  * [5] S. Ganji, K. Dhawan, and R. Sinha, ``IITG-Hingcos corpus: A Hinglish code-switching database for automatic speech recognition,'' _Speech communication_ , vol. 110, pp. 76–89, 2019.
  * [6] S. Baghel, S. Ramoji, Sidharth, R. H, P. Singh, S. Jain, P. Roy Chowdhuri, K. Kulkarni, S. Padhi, D. Vijayasenan, and S. Ganapathy, ``The DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments,'' in _Proc. INTERSPEECH 2023_ , 2023, pp. 3562–3566.
  * [7] ``Rich transcription evaluation,'' https://www.nist.gov/itl/iad/mig/rich-transcription-evaluation, accessed: 2024-02-29.
  * [8] J. G. Fiscus, J. Ajot, and J. S. Garofolo, ``The rich transcription 2007 meeting recognition evaluation,'' in _International Evaluation Workshop on Rich Transcription_. Springer, 2007, pp. 373–389.
  * [9] N. Ryant, P. Singh, V. Krishnamohan, R. Varma, K. Church, C. Cieri, J. Du, S. Ganapathy, and M. Liberman, ``The Third DIHARD Diarization Challenge,'' in _Proc. Interspeech 2021_ , 2021, pp. 3570–3574.
  * [10] A. Nagrani, J. S. Chung, J. Huh, A. Brown, E. Coto, W. Xie, M. McLaren, D. A. Reynolds, and A. Zisserman, ``Voxsrc 2020: The second voxceleb speaker recognition challenge,'' _arXiv preprint arXiv:2012.06867_ , 2020.
  * [11] A. Joglekar, J. H. Hansen, M. C. Shekar, and A. Sangwan, ``FEARLESS STEPS Challenge (FS-2): Supervised Learning with Massive Naturalistic Apollo Data,'' in _Proc. Interspeech 2020_ , 2020, pp. 2617–2621.
  * [12] H. Liu, H. Xu, L. P. Garcia, A. W. Khong, Y. He, and S. Khudanpur, ``Reducing language confusion for code-switching speech recognition with token-level language diarization,'' in _2023 IEEE ICASSP_. IEEE, 2023, pp. 1–5.
  * [13] S. V, V. Thenkanidiyoor, and D. A. D, ``SVM Based Language Diarization for Code-Switched Bilingual Indian Speech Using Bottleneck Features,'' in _Proc. SLTU 2018_ , 2018, pp. 132–136.
  * [14] V. Y. H. Chua, H. Liu, L. P. Garcia, F. T. Woon, J. Wong, X. Zhang, S. Khudanpur, A. W. H. Khong, J. Dauwels, and S. J. Styles, ``MERLIon CCS Challenge: A English-Mandarin code-switching child-directed speech corpus for language identification and diarization,'' in _Proc. INTERSPEECH 2023_ , 2023, pp. 4109–4113.
  * [15] T. Schultz and A. Waibel, ``Language-independent and language-adaptive acoustic modeling for speech recognition,'' _Speech Communication_ , vol. 35, no. 1-2, pp. 31–51, 2001.
  * [16] B. M. L. Srivastava, S. Sitaram, R. Kumar Mehta, K. Doss Mohan, P. Matani, S. Satpal, K. Bali, R. Srikanth, and N. Nayak, ``Interspeech 2018 Low Resource Automatic Speech Recognition Challenge for Indian Languages,'' in _Proc. SLTU 2018_ , 2018, pp. 11–14.
  * [17] ``Interspeech 2018 special session : Speech recognition for Indian languages,'' https://sites.google.com/view/interspeech2018-ss1, accessed: 2024-02-29.
  * [18] ``Hindi ASR challenge,'' https://sites.google.com/view/asr-challenge, accessed: 2024-02-29.
  * [19] ``Hindi-Tamil-English ASR challenge,'' https://sites.google.com/view/indian-language-asrchallenge/, accessed: 2024-02-29.
  * [20] A. Diwan, R. Vaideeswaran, S. Shah, A. Singh, S. Raghavan, S. Khare, V. Unni, S. Vyas, A. Rajpuria, C. Yarra _et al._ , ``Multilingual and code-switching asr challenges for low resource indian languages,'' _arXiv preprint arXiv:2104.00235_ , 2021.
  * [21] ``Multilingual and code-switching speech recognition,'' https://www.clsp.jhu.edu/multilingual-and-code-switching/, accessed: 2024-02-29.
  * [22] J. Li, B. Wang, Y. Zhi, Z. Li, L. Li, Q. Hong, and D. Wang, ``Oriental language recognition (olr) 2020: Summary and analysis,'' _arXiv preprint arXiv:2107.05365_ , 2021.
  * [23] S. Baghel, S. Ramoji, S. Jain, P. R. Chowdhuri, P. Singh, D. Vijayasenan, and S. Ganapathy, ``Summary of the displace challenge 2023–diarization of speaker and language in conversational environments,'' _arXiv preprint arXiv:2311.12564_ , 2023.
  * [24] ``SCTK, the NIST scoring toolkit,'' https://github.com/usnistgov/SCTK/, accessed: 2024-02-29.
  * [25] D. Snyder, D. Garcia-Romero, G. Sell, A. McCree, D. Povey, and S. Khudanpur, ``Speaker recognition for multi-speaker conversations using x-vectors,'' in _ICASSP 2019-2019 IEEE International conference on acoustics, speech and signal processing (ICASSP)_. IEEE, 2019, pp. 5796–5800.
  * [26] P. Singh, H. Vardhan, S. Ganapathy, and A. Kanagasundaram, ``LEAP diarization system for the second DIHARD challenge,'' in _Proc INTERSPEECH_ , 2019, pp. 983–987.
  * [27] H. Bredin, ``pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe,'' in _Proc. INTERSPEECH 2023_ , 2023, pp. 1983–1987.
  * [28] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, ``Robust speech recognition via large-scale weak supervision,'' in _International Conference on Machine Learning_. PMLR, 2023, pp. 28 492–28 518.
  * [29] J. Valk and T. Alumäe, ``VoxLingua107: a dataset for spoken language recognition,'' in _Proc. IEEE SLT Workshop_ , 2021.
  * [30] B. Vachhani, D. Singh, and R. Lawyer, ``Multi-resolution Approach to Identification of Spoken Languages and To Improve Overall Language Diarization System Using Whisper Model,'' in _Proc. INTERSPEECH 2023_ , 2023, pp. 1993–1997.
  * [31] F. Landini, J. Profant, M. Diez, and L. Burget, ``Bayesian HMM clustering of x-vector sequences (vbx) in speaker diarization: theory, implementation and analysis on standard tasks,'' _Computer Speech & Language_, vol. 71, p. 101254, 2022.
  * [32] ``Turn speech into text using google ai,'' https://cloud.google.com/speech-to-text?hl=en, accessed: 2024-02-29.
  * [33] S. Watanabe, M. Mandel, J. Barker, E. Vincent, A. Arora, X. Chang, S. Khudanpur, V. Manohar, D. Povey, D. Raj _et al._ , ``Chime-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,'' _arXiv preprint arXiv:2004.09249_ , 2020.
  * [34] J. Kalda, R. Marxer, T. Alumäe, H. Bredin _et al._ , ``Pixit: Joint training of speaker diarization and speech separation from real-world multi-speaker recordings,'' _arXiv preprint arXiv:2403.02288_ , 2024.
  * [35] H. Wang, C. Liang, S. Wang, Z. Chen, B. Zhang, X. Xiang, Y. Deng, and Y. Qian, ``Wespeaker: A research and production oriented speaker embedding learning toolkit,'' in _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2023, pp. 1–5.
  * [36] F. Jia, S. Majumdar, and B. Ginsburg, ``Marblenet: Deep 1d time-channel separable convolutional neural network for voice activity detection,'' in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6818–6822.
  * [37] L. Barrault, Y.-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Duppenthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheim _et al._ , ``Seamless: Multilingual expressive and streaming speech translation,'' _arXiv preprint arXiv:2312.05187_ , 2023.
  * [38] A. Gusev, V. Volokhov, T. Andzhukaev, S. Novoselov, G. Lavrentyeva, M. Volkova, A. Gazizullina, A. Shulipa, A. Gorlanov, A. Avdeeva, A. Ivanov, A. Kozlov, T. Pekhovsky, and Y. Matveev, ``Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances,'' in _Proc. The Speaker and Language Recognition Workshop (Odyssey 2020)_ , 2020, pp. 179–186.
  * [39] V. S. Lodagala, S. Ghosh, and S. Umesh, ``Ccc-wav2vec 2.0: Clustering aided cross contrastive self-supervised learning of speech representations,'' in _2022 IEEE Spoken Language Technology Workshop (SLT)_. IEEE, 2023, pp. 1–8.


# Testing optomechanical microwave oscillators for SATCOM application

Laura Mercadé, Eloy Rico, Jesús Ruiz Garnica, Juan Carlos Gómez, Amadeu Griol,
Miguel A. Piqueras, Alejandro Martínez and Vanessa C. Duarte L. Mercadé, J.C.
Gómez, A. Griol and A. Martínez are with the Nanophotonics Technology Center,
Universitat Politècnica de València, Valencia 46022, Spain and E. Rico, J.
Ruiz Garnica, M. A. Piqueras and V. C. Duarte are with DAS Photonics, Valencia
46022, Spain.

###### Abstract

The realization of photonic microwave oscillators using optomechanical
cavities has recently become a reality. By pumping the cavity with a blue-
detuned laser, the so-called phonon lasing regime - in which a mechanical
resonance is amplified beyond losses - can be reached and the input signal
gets modulated by highly-coherent tones at integer multiples of the mechanical
resonance. Implementing optomechanical cavities on released films with high
index of refraction can lead to optical modes at telecom wavelengths and
mechanical resonances in the GHz scale, resulting in highly-stable signals in
the microwave domain upon photodetection. Owing to the extreme compactness of
such cavities, application in satellite communications (SATCOM) seems highly
appropriate, but no experiments have been reported so far. In this paper, an
optomechanical microwave oscillator (OMO) built on a micron-scale silicon
optomechanical crystal cavity is characterized and tested in a real SATCOM
testbed. Using a blue-detuned laser, the OMO is driven into a phonon lasing
state where multiple harmonics are generated, reaching tones up to 20 GHz.
Under this regime, its practical applicability, remarkably addressing its
performance as a photonic local oscillator, has been validated. The results,
in addition with the advantages of extreme compactness and silicon-technology
compatibility, make OMOs very promising candidates to build ultra-low weight
photonics-based microwave oscillators for SATCOM applications.

###### Index Terms:

optomechanical cavity, phonon lasing, microwave oscillator, SATCOM
application, silicon photonics.

##  I Introduction

Photonic technology is becoming a key ingredient of the most advanced SATCOM
payloads, greatly upgrading their performance thanks to the intrinsic
advantages of photonics over electronics: low weight, compactness, low-power
consumption, broadband performance and immunity to electromagnetic
interference, among others. Remarkably, the most important benefits of
photonic payload in SATCOM systems have been demonstrated [1, 2, 3] in the
framework of European H2020 OPTIMA project [4]. As is well known, a local
oscillator (LO) is a key building block in microwave -and photonic- satellite
payloads based on typical superheterodyne receiver [5]. In photonics, this is
implemented by an optoelectronic oscillator (OEO) [6] like the SATCOM payload
module named photonic frequency generation unit (PFGU) [5]. Its function is,
essentially, to efficiently convert the LO microwave signal into the optical
domain.

Figure 1(a) shows the block diagram and the resulting optical LO signal of a
typical PFGU, which includes OPTIMA PFGU, where an electrical (or RF) LO
externally modulates an optical carrier generated by a continuous wave (CW)
laser using an electro-optical modulator (EOM), whose output is amplified up
to the desired level by an optical amplifier (OA). The main drawbacks of this
approach are the need for an electrical oscillator (these LOs are bulky and
power consuming) requiring conversion to the optical domain as well as and the
need for costly and bulky EOMs, which also require complex control electronics
to select and maintain the right operating point.

Figure 1: (a) Block diagram of PFGU (OEO) of a SATCOM photonic payload. (b)
All-optical PFGU approach using an OMO driven to the phonon lasing regime. The
used acronyms stands for CW: continuous wave, EOM: electro optical modulator,
LO: local oscillator, OA: optical amplifier, PD: photodetector, RSA: RF
spectrum analyzer, OMO: optomechanical microwave oscillator.

Alternatively, one may use a fully-optical device as PFGU. The block diagram
of this approach, as well as the output optical LO signal, is depicted in Fig.
1(b). The novel element is an optomechanical cavity [7], which under the
appropriate conditions can perform as a photonic LO [8] within a foot-print of
several square microns, thus being suitable for massive integration in silicon
wafers. In this kind of cavity, optical and mechanical waves can be
simultaneously localized and, as a result of different surface and volume
acousto-optic effects [9], interact strongly, which gives rise to intriguing
phenomena. Indeed, upon blue-detuned laser driving, dynamical back-action in
the cavity [10] can lead to self-sustained oscillations, also known as phonon
lasing [11]. In this regime, the input laser gets modulated by a very narrow
tone so the device can perform as a photonic microwave oscillator with
frequencies determined by the mechanical resonance(s) of the cavity [12, 13,
8]. Under stronger pumping, the input laser signal gets modulated by a set of
narrow peaks at integer multiples of the mechanical resonance frequency,
giving rise to the optomechanical (OM) equivalent of an optical frequency comb
(OFC) [8].

However, despite its appealing features as photonic microwave oscillator [14,
15], OMO devices have not yet been tested when forming part of complex system
used in real applications. In this work, we validate the performance of a
silicon-chip OMO having a fundamental mechanical oscillation frequency around
4 GHz as a photonic LO in a real SATCOM test bed, thus unveiling the potential
to use OM devices out of the lab.

##  II OM cavity characterization

Here we describe the silicon OM cavity that we use in our experiments (see
Fig. 2(a)). As reported in [8], the OM cavity essentially consists of a
released silicon nanobeam that is structured so that at its sides there are
periodic photonic crystal mirrors that provide a photonic bandgap for TE-like
modes at the desired wavelengths. The unit-cell dimensions are adiabatically
modified when moving from the cavity sides to its center - for instance, the
width of the lateral wings is diminished - resulting in localized photonic
states with large optical Q factor. This will also result in localized
mechanical states [16], which can interact with the confined optical modes.

An important point is that we cannot choose the mechanical frequencies at
will. Instead, they are constrained by our choice of photonic cavity which
must resonate at 1550 nm in order to use telecom optical equipment for
characterization. Since the light speed is about five orders of magnitude
higher than the speed of sound in silicon, mechanical resonances will occur at
frequencies five orders of magnitude lower than the optical frequencies [9].
Therefore, OM cavities in silicon nanobeams have mechanical modes with
frequencies in the scale of several GHz. Moreover, usually a lot of mechanical
modes, with different localization patterns will coexist [17] (notice that
extended modes with frequencies of tens of MHz will also be present [18]). In
our cavity, we get a set of GHz-scale mechanical modes confined in the defect
region [19]. However, according to experiments, the mode with the largest OM
interaction - see below - is the one which oscillates at a mechanical
frequency $f_{m}=\Omega_{m}/2\pi$ =3.874 GHz [8]. Other OM cavities in silicon
nanobeams can lead to more mechanical frequencies in the GHz regime, even
reaching frequencies above 10 GHz through more sophisticated designs [20].

From the optical and mechanical field patterns, which can be obtained via
numerical simulations [21], it is possible to calculate the vacuum OM coupling
rate, $g_{0}=g_{OM}\sqrt{\hbar/2m_{eff}\Omega_{m}}$, which measures the shift
of the optical frequency due to the fluctuations of the ground state of the
mechanical oscillator [7, 9]. In the last definition, $\hbar$ is the reduced
Planck constant and $m_{eff}$ is the effective mass of the cavity. Such
coupling rate, $g_{OM}$, has two main contributions, the moving interface (MI)
and photoelastic (PE) effect, which model respectively surface and volumetric
OM interaction, resulting in $g_{OM}=g_{MI}+g_{PE}$. The MI term can be
calculated as [22]:

${\color[rgb]{0,0,0}g_{MI}=-\frac{\omega}{2}\frac{\oint(\textbf{U}\cdot\textbf{\^{n}})(\Delta\varepsilon|\textbf{E}_{\parallel}|^{2}-\Delta\varepsilon^{-1}|\textbf{D}_{\perp}|^{2}dS)}{\int\textbf{E}\cdot\textbf{D}dV}}$
(1)

where U is the normalized displacement field (max$|\textbf{U}|$=1), n̂ is the
outward pointing normal to the boundary,
$\Delta\varepsilon=\varepsilon_{Si}-\varepsilon_{Air}$ and
$\Delta\varepsilon^{-1}=\varepsilon_{Si}^{-1}-\varepsilon_{Air}^{-1}$ [21].
Here, $\varepsilon_{Si}$ and $\varepsilon_{Air}$ accounts for the silicon and
air permittivity, respectively.

The PE coupling rate can be calculated as:

${\color[rgb]{0,0,0}g_{PE}=-\frac{\omega}{2}\frac{\langle
E|\delta\varepsilon|E\rangle}{\int\textbf{E}\cdot\textbf{D}dV}}$ (2)

with $\delta\varepsilon=-\varepsilon_{0}n^{4}p_{ijkl}S_{kl}$. Here,
$\varepsilon_{0}$ is the vacuum permittivity, $n$ the refractive index of
silicon, $p_{ijkl}$ are the photoelastic tensor components and $S_{kl}$ the
strain tensor components.

The vacuum OM coupling rate is one of the key parameters of an OM cavity: the
higher it is, the more efficient all the underlying OM phenomena will be. In
our case, we get $g_{0}/2\pi$ = 540 kHz [8], not far from the best values
reported in literature [20, 23].

Essentially, under the appropriate conditions (blue-detuned driving to achieve
phonon lasing), at the output of the OM cavity we will have an optical carrier
(provided by the driving laser and with frequency $\omega_{L}$) modulated by a
set of harmonics spaced by the mechanical frequency. Each one of these peaks
will have a certain amplitude or power, which will determine the available
microwave power $P_{LO,n}$ at a certain frequency $f_{LO,n}=n\omega_{m}/2\pi$.
As shown below, these microwave tones will display very low phase noise, in
particular the 1st harmonic [8].

Figure 2: (a) Scanning electron microscopy image of a single OM crystal cavity
as the one used in this work. (b) Optical microscope top-view of the coupling
between a tapered fiber loop close to the OM cavity. (c) Block diagram of the
experimental setup used in the OM characterization of the cavities.

Once light is coupled into the cavity via a tapered fiber loop (see Fig.
2(b)), OM interaction will transduce the mechanical modes into the optical
signal, being then detected using a photodetector. The full experimental setup
to measure the OM cavity is presented in Fig. 2(c) (the loop and the cavity
are not to scale). Here, a continuous wave (CW) is generated with a tuneable
fiber-coupled external cavity diode laser (New Focus Velocity TLB-6728). Then,
the optical signal is amplified by means of an erbium doped fiber amplifier
(EDFA) to fulfil our power requirements, and a polarization controller (PC)
sets the required polarization that will finally arrive at the device under
characterization. Driving the cavity requires to have the incident field
parallel to the chip plane so that a transverse electric (TE) mode of the
cavity is excited [8]. After the polarization controller the optical wave is
sent through a circulator that generates two paths: transmission (port 2) and
reflection (port 3). To ensure a closed environment that minimizes external
interferences, the whole system is placed inside a methacrylate cage structure
where both the tapered micro-looped fiber and the sample are located.

When the laser is switched on, the light travels along the fiber and couples
to the cavity -via evanescent field coupling- provided that the loop is close
enough to it. The relative position between these two elements can be
accurately tuned with piezoelectric controllers placed at the sample holder
where the OM cavity is located. Then the cavity can scatter lights back into
the fiber, being guided along both the transmission and reflection direction.
Regarding the transmission channel, we can switch between two different
configurations as a function of the measurement to be performed. For optical
spectrum measurements, the signal is first amplified (if needed) with an EDFA
and then it is analyzed through an optical spectrum analyzer (OSA) ANDO
AQ6317C. The optical resonance measurements are recorded using an alternative
configuration. In this case, the signal is attenuated with a variable optical
attenuator (VOA), and then it is photo detected via a switchable gain
amplified detector (Thorlabs PDA20CS-EC - InGaAs switchable gain amplified
detector) and finally recorded with a scope (DSO81204B Infinium high
performance oscilloscope).

The reflection channel ends in either a 12 GHz-bandwidth photodetector (New
Focus 1544-B DC-coupled NIR fiber-optic receiver) or a 50 GHz-bandwidth
photodetector (u2XPDV2120R). It must be noted that whilst the 12 GHz bandwidth
detector has more sensitivity, the second one permits us to measure higher-
order harmonics of the fundamental mechanical resonance. Once the signal has
been photodetected we can choose either to analyze it through an RF spectrum
analyzer (RSA) or use it as the trigger signal for the temporal trace
measurements. It must be remarked that different RSA have been employed in our
experiments: one for the analysis of the mechanical response of the cavity
(Rohde $\&$ Schwarz FSQ 40) and another for the phase noise measurements
(Rohde $\&$ Schwarz FSUP signal source analyzer or Anritsu MS2850A signal
analyzer).

##  III Optomechanical microwave oscillator (OMO)

When the cavity is driven by a low-power laser whose wavelength is blue-
detuned with respect to the cavity resonance, the thermal motion of the cavity
is transduced into the optical signal and can be detected in the
photodetectors. An example of the transduced mechanical motion response before
reaching the instability threshold is presented in Fig. 3(a), which represents
the thermal phonons present in the cavity and moving at frequencies around the
mechanical frequency $f_{m}$, which in this particular device was
$f_{m}\approx$ 3.894 GHz. The spectrum in Fig. 3(a) has been labelled as OMO
OFF because the mechanical mode is simply transduced but has not yet reached
the self-oscillations regime. When driving the cavity with a blue-detuned
laser with enough power, the overall mechanical damping rate is highly
reduced. By these means, the mechanical cavity dynamics evolves and
amplification of the mechanical motion can be obtained as a result of
dynamical back-action [11]. Indeed, a threshold of instability can be attained
once the mechanical losses equate to the optical antidamping and the
oscillator reaches a condition of self-oscillation (also known as phonon
lasing regime). That results in a very narrow tone in the detected spectrum at
the mechanical resonance as shown on Fig. 3(b) (labelled as OMO ON), which
depicts the electrical spectrum around the central frequency of the detected
peak (termed $f_{LO}$ hereinafter) once we have reached the phonon lasing
condition. The narrowing of the detected signal, as well as the increase of
the power, both signatures of phonon lasing, can be easily noticed. This is a
direct proof that the cavity can be employed as an OMO. It must be highlighted
that despite its equivalence with the OEO in terms of the functions it can
provide, here, the involved mechanism is a self-sustained oscillation
originated from OM interaction.

Figure 3: (a) Power spectral density of the transduced mechanical spectrum
when the OMO is off (OMO OFF) and the tranducd signal comes from the thermal
occupation. (b) PSD spectrum of the first harmonic when the phonon lasing
state is reached (OMO ON). (c) Recorded optical spectrum when the OMO is
activated or not (in green and blue, respectively) showing a set of peaks
corresponding to different harmonics represented (in red) at the expected
position. (d) PSD spectrum of the OFC generated with the OMO.

Notice that when back-action effects take place, the detected frequency in the
electrical spectrum can be different from the original mechanical frequency.
This shift is a result of the optical spring shift effect [20, 11], which
changes the stiffness of the silicon nanobeam according to the total amount of
power in the cavity and the laser-cavity detuning. This means that in the
phonon lasing regime $f_{LO}$ will slightly differ from $f_{m}$, which in real
applications can be used for dynamic tuning of the oscillator. In the cavity
under study, the variation range of $f_{LO}$ can be up to 5 MHz once the
cavity is in the lasing regime [8, 24].

Indeed, under sufficiently strong pumping [8], the optical carrier becomes
modulated by a set of harmonics spaced by $f_{LO}$ thus generating an optical
frequency comb of OM nature. At this operation point, labelled as OMO ON, the
resulting optical spectrum of the transmitted signal is depicted in Fig. 3(c).
Here, a noticeable difference between the optical response when the OMO is
activated (blue curve) or not (green curve) can be appreciated. In the case
when the mechanical motion is only transduced - OMO OFF case - only two
sideband peaks spaced $\pm f_{m}$ from the CW laser signal are observed. Once
the cavity is in the self-sustained regime, different sidebands (depicted in
red arrows) at frequencies $n\times f_{LO}$ around the laser frequency, where
$n$ is the harmonic index, are generated. If we analyze the RF spectrum or
power spectral density (PSD) of the system when the OMO is active, obtained
upon photodetection of the previously analyzed optical signal, higher-order
harmonics appear, as presented in Fig. 3(d) for the first five harmonics.

A key aspect determining the quality of microwave oscillators is the presence
of random fluctuations in the phase of the generated tone. In the frequency-
domain representation, these fluctuations can be measured by the phase noise,
which is related to the time domain deviations from the case of an ideal
signal with zero jitter. In our case, once the cavity is in the lasing regime,
the generated microwave tone can be characterized by these means. The phase
noise of the different harmonics generated by driving the cavity in the lasing
state was reported in [8], showing a behavior similar to that of standard
harmonic mixers (the phase noise grows with the order of the harmonic).
However, whilst the phase noise is a powerful tool to analyze the short-term
stability of the oscillator, the characterization of its long-term stability
is also needed for practical applications. Such long-term stability is usually
quantified by the Allan deviation $\sigma_{y}(\tau)$, which provides
information about the frequency drift of the oscillator [25, 26, 27].

The Allan variance $\sigma_{y}^{2}(\tau)$ depends on the variable $\tau$ and
expresses the mean square of all frequency counter samples separated in time
by $\tau$ over the entire measurement interval [25]. The resulting
$\sigma_{y}(\tau)$ is presented at Fig. 4. Here, different noise sources as
the flicker frequency noise ($\sigma_{y}(\tau)\approx\tau^{0}$) and a
contribution of the random walk of frequency
($\sigma_{y}(\tau)\approx\tau^{1}$) can be appreciated. The Allan deviation
has been calculated through:

$\sigma_{y}(\tau)=\sqrt{\int_{0}^{\infty}\frac{4f^{2}L(f)}{f_{c}^{2}}\frac{\sin^{4}(\pi\tau
f)}{(\pi\tau f)^{2}}df}$ (3)

where $L(f)$ is the measured phase noise, $\tau$ the averaging time and
$f_{c}$ the carrier frequency of the oscillator under study [19, 26]. The
derived Allan deviation through the phase noise data is in agreement with the
power-law fits (in dashed lines in Fig. 4 obtained via the conversion
relationships in [25], thus resulting in a good approximation. On the other
side, a broader analysis of the long-term stability of this system is
presented in Ref. [24].

Figure 4: Allan deviation results converted from the measured phase noise
results for the OMO under study (blue continuous line) and its power law fits
contributions (dashed lines).

To further assess the stability of the detected electrical signal for its use
as an LO, the long-term signal stability was measured. This measurement was
carried out by letting on trace in max-hold mode, so it maintains the maximum
power value for the rest of the test. Figure 5(a) shows the power detected for
first harmonic over a 3 hours time window measurement. Also, the single-
sideband (SSB) phase noise was obtained (blue curve) using the phase noise
measurement application of the spectrum analyzer. As a reference, an ultra-low
phase noise RF generator was employed (green curve). Figure 5(b) shows both
responses. Two different OMO configurations were measured to discard phase
noise high dependence on each one. It should be considered that the OMO is
under different configuration since it was measured under different
conditions, such as: the wavelength detuning or input power. The phase noise
level at a 100 kHz offset frequency shown in Fig. 5(b) is consistent with the
phase noise obtained in [8] for a similar device.

Figure 5: (a) Maximum held trace showing the amplitude stability and temporal
frequency drift of the 1st harmonic during a 3 hours measurement. (b) Phase
noise (PN) level of OMO (blue trace) versus a commercial low PN microwave
generator (green trace).

##  IV OMO integration in a SATCOM testbed

Once the OMO has been validated and its basic performance has been
characterized, it is time to assess its practical applicability on a real
space-application testbed, like in the context of the OPTIMA project [4],
where the main goal was demonstrating the strengths of photonic payloads, as
reported before. The architecture of photonic satellite payload includes LO
generation (with the requirements of high power and low phase noise [28]),
frequency down conversion (DOCON) and photonic receiver, with intermediate
frequency (IF) as an output [1]. The reference specifications of OPTIMA
project are summarized in Table I. In our case, the goal is to test the
performance of the OMO device presented in Ref. [8] within the OPTIMA system.

TABLE I: OPTIMA test bed reference specification. Parameter | Requirement
---|---
Gain (dB) |  $<$ -17
Noise Figure (dB) |  $<$ 42
Phase Noise (dBc/Hz) |
10 Hz |  $<$ -39
100 Hz |  $<$ -73
1 kHz |  $<$ -85
10 kHz |  $<$ -96
100 kHz |  $<$ -105
1 MHz |  $<$ -114
10 MHz |  $<$ -131
LO Harmonics (dBc) |  $<$ -40 max
Intermodulation products (dBc) |  $<$ -65 max
Figure 6: Integration of the OMO device in the OPTIMA testbed. The used
acronyms stands for CW: continuous wave, OMO: optomechanical microwave
oscillator, OA: optical amplifier, PD: photodetector, RSA: RF spectrum
analyzer, PC: polarization controller, EOM: electro optical modulator, OEM:
opto electronic module, OMC: opto mechanical cavity, PFGU: photonics frequency
generation unit, PMFC: photonic multi-frequency converter, TM/TC: telemetry /
telecommand, SM: single mode, PM: polarization-maintaining and EGSE:
electrical ground support equipment.

The testbed is schematically shown in Fig. 6, where the OMO is used to
generate LO signal in an all-optical way. A polarization controller (PC) is
introduced between the OMO and the DOCON named PMFC (photonic multi-frequency
converter) unit to properly align the correct polarization axis to the Mach-
Zehnder EOM inside the PMFC. Notice that the OMO uses a single mode fiber but
polarization-maintaining fiber is needed at the input of EOM. Then, an optical
amplifier overcomes the power limitation of the OMO (there is a maximum power
that we can inject into the OM cavity without damaging the material). Next, a
5dB optical attenuator is placed between the PMFC output fiber and the the
receiver named OEM (optoelectronic module) to prevent optical overpower. Also,
the second channel of the PMFC is connected as a power monitor. For the sake
of simplicity, since only one wavelength is employed, both
multiplexing/demultiplexing unit and the switching matrix are not mounted on
the validation testbed.

Before showing the functional results, it is important to know the limitations
of the OMO in terms of power availability and modulation depth. Figure 7 shows
a comparison of the output optical field of both the OMO experiment and the
OPTIMA PFGU (also named photonic local oscillator, PhLO, Fig. 1(a)).
Specifically, Fig. 7(a) shows the optical spectrum of the photonic LO from the
SATCOM system whereas Fig. 7(b) depicts the optical spectrum from the OMO when
operated to generate an OFC.

Figure 7: (a) Optical spectrum (in red) at OPTIMA PhLO output. (b) Optical
spectrum (in red) at OMO output. Vertical lines represent the different
harmonics.

After analyzing such figures, three important aspects must be highlighted. The
first is that the OMO delivers less total optical power than the OPTIMA PhLO
since the carrier and the first-order sideband give +8 dBm and -8 dBm,
respectively. On the other hand, the PhLO gives +10 dBm on the carrier and
+4.5 dBm on the first-order sideband. Thus, maintaining the rest of the
parameters equal for both configurations, higher conversion gain is expected
from the OPTIMA PhLO than in the OMO configuration. The second is that the
OPTIMA PhLO exhibits a higher modulation efficiency than the OMO, given the
fact that the first-order sidebands have more power density than the same
sidebands in the OMO. This also contributes to a higher conversion gain in the
case of the PhLO. The third observation is that the OMO is more non-linear
than the PhLO. This is because the PhLO uses an optical modulator with a
medium modulation depth and its behaviour is quite linear, providing most of
the modulating power to the fundamental harmonic signal (first sideband). On
the contrary, the OMO has a more non-linear response, generating more
sidebands so that the power is more distributed among all the harmonic levels.
This means that the OMO will beat the OPTIMA PhLO architecture (keeping the
rest of the parameters equal in both architectures) when addressing frequency
conversion beyond the first harmonic.

##  V Test results

This section presents the test results of the OMO when mounted in the OPTIMA
testbed as described before. Three different configurations are measured to
perform a full test of the device as a photonic local oscillator. In all of
them, the input signal to PMFC is a 29.0 GHz RF tone with -8 dBm power. An
example of the optical power spectrum of the OMO once it is driven through the
OPTIMA testbed can be seen in Fig. 8

Figure 8: Recorded optical spectrum (in red) when the OMO is sent via the
SATCOM OPTIMA test bench. Vertical lines represent the different harmonics.

As reported before, this OMO architecture is highly dependent on environmental
conditions of the room where test bed is installed, so each time the OMO is
switched on, its functional parameters exhibit a slight variation with respect
to the previous measurement time. This dependence on the environment  can
strongly affect the excitation conditions, since the fiber taper that excites
the OM cavity can suffer from fluctuations induced by changes in temperature,
pressure, or humidity. Notably, improving the coupling mechanisms and getting
a packaged device would enormously reduce the environmental dependence,
improving the overall performance of the OMO.

Figure 9: Output RF spectra for (a) configuration 1 at 1554,59 nm, (b)
configuration 2 at 1554,59 nm and (c) configuration 3 at 1558,75 nm.

To provide a reference for the functional tests and the testbed, a set of
simulations $–$ using experimental data $-$ was carried out. These simulations
consider the performance of each key component at the photonic conversion sub-
unit, as well as the known functional parameters already measured on the OMO
subsystem (operational wavelength that depends on OMO final stable
configuration, LO optical output power that is also dependent on the
configuration and the pump power, LO fundamental frequency that depends on the
cavity mechanical response, and LO harmonic levels that are dependent on the
final configuration). Once the LO optical field is modeled, this signal is
amplified by using the physical expressions of EDFA. Then, this LO amplified
signal is modulated by using the theoretical expressions of a Mach-Zehnder EOM
(insertion loss and half wave voltage ($V_{\pi}$), with a bias voltage as a
trade-off between high gain, low noise, and good linearity). Afterwards, two
cascaded amplifiers are simulated (the signal is amplified again to boost the
signal level and compensate optical losses). Finally, the signal is detected
by using a broadband RF photodiode (with responsivity, bandwidth and $S_{21}$
response given by the datasheet; and 50 Ohm impedance on all RF interfaces).
All the harmonic and intermodulation products are located along the electrical
spectrum to obtain the electrical power of each signal.

It must be noted that the OMO has an optical resonance around 1522 nm with an
optical quality factor up to $10^{3}$ [8]. However, when the driving power is
sufficiently large (as required to reach the phonon lasing regime), the large
thermo-optic effect of silicon gives rise to bistability with a “saw-tooth”
shaped transmission, thus red-shifting the resonance frequency [11]. Under
this driving condition, the different configurations of the OMO were studied.
In configuration 1 the OMO was stabilized at a wavelength of 1554,59 nm and
with an input driving power of 11 mW. This configuration was lost after 3-hour
measurement. Then, the OMO was stabilized again at a driving wavelength of
1554,59 nm and with an input power of 11 mW (configuration 2). At other
measurement time, in configuration 3, the system was stabilized at a
wavelength of 1558,75 nm and an OMO input power equal to 13 mW. The measured
output spectrum (for the three configurations) are shown in Fig. 9 and
summarized in Table II (together with simulations results of configuration 2).

Figure 10: (a) AM noise on IF output signal. (b) AM noise on output RF signal.

Besides the spectrum analysis and the intermodulation products, configuration
3 testing was complemented with additional measurements over some output
products to assess the amplitude and frequency noise. To this end, the
spectrum analyzer was configured as both amplitude modulation (AM) and
frequency modulation (FM) analyzer. Figures 10 and 11 show some interesting
results of RF, LO and intermediate frequency (IF) output products. Notice
that, in the context of a superheterodyne receiver, the signals are
transmitted at the RF frequency and processed at the much-lower IF frequency,
being both related by the LO frequency as $f_{LO}=f_{RF}-f_{IF}$.

TABLE II: Test results of three configurations and simulated values of configuration 2. Product | Freq | Power | Power | Power | Power sim. | Conversion | Conversion | Conversion | Conversion
---|---|---|---|---|---|---|---|---|---
| (GHz) | (dBm) | (dBm) | (dBm) | (dBm) | gain (dB) | gain (dB) | gain (dB) | gain sim. (dB)
|  | Conf. 1 | Conf. 2 | Conf. 3 |  | Conf. 1 | Conf. 2 | Conf. 3 |
RF | 29 | -25.3 | -22,8 | -18,1 | -23.2 | -17.3 | -14,8 | -10,1 | -15.2
LO | 3,888 | -33,8 | -33,4 | -26,4 | -26,4 | - | - | - | -
2LO | 7,777 | -45,1 | -36,7 | -39,8 | -50,1 | - | - | - | -
3LO | 11,666 | -58,0 | -54,5 | -63,2 | -86,3 | - | - | - | -
4LO | 15,554 | -55,6 | -52,1 | -61,4 | -114 | - | - | - | -
5LO | 19,443 | -71,7 | -58,4 | -69,6 | -123 | - | - | - | -
RF-LO | 25,111 | -39,3 | -35,4 | -32,8 | -33,4 | -31,3 | -27,4 | -24,8 | -25,5
RF-2LO | 21,222 | -52,3 | -47 | -46,2 | -57,0 | -44,3 | -39,0 | -38,2 | -49,0
RF-3LO | 17,333 | -65,4 | -56,9 | -70,4 | -93,4 | -57,4 | -48,9 | -62,4 | -85,4
RF-4LO | 13,445 | -70,1 | -59,1 | -72,1 | -120,5 | -62,1 | -51,1 | -64,1 | -112,5
RF-5LO | 9,556 | -86,7 | -72,3 | -79,4 | -119,3 | -78,7 | -64,3 | -71,4 | -11,3
RF+LO | 32,888 | -45,2 | -42,4 | -39,6 | -33,4 | -37,2 | -34,4 | -31,4 | -25,5
RF+2LO | 36,777 | -63,3 | -58,4 | -58,1 | -57,1 | -55,7 | -50,4 | -50,1 | -49,1
RF+3LO | 40,666 | -78,7 | -65,6 | -82,6 | -93,6 | -70,7 | -70,7 | -74,6 | -85,6
RF+4LO | 44,554 | -76,4 | -62,9 | -77,6 | -113,0 | -68,4 | -68,4 | -69,6 | -105,0

To evaluate the AM noise, Fig. 10(a) depicts the amplitude distribution of the
down-converted IF signal. The mean value is -33.09 dBm with a crest factor
(peak power - mean power) of 1.36 dB, which is an important value since it is
an indicator of amplitude instability in the output signal. Next, the same
amplitude distribution is shown for the RF output signal in Fig. 10(b). Here,
the mean value is -19.04 dBm with a crest factor of 0.15 dB, which is a more
acceptable value. It is an indicator of amplitude stability in the output
signal. An early conclusion can be drawn at this point: the main amplitude
instability comes from the LO signal, since the mixed IF signal (RF-LO) has
more instability than non-mixed output signal (RF).

Another important measurement is the FM noise of the output products. Figure
11(a) shows the main FM parameters of the LO first harmonic at the output for
the three different configurations. This noise was measured during a short
time window (short term), typically less than two minutes observation time.
The root mean square (RMS) FM noise is 505.3 Hz and the peak-to-peak parameter
is 2.8 kHz. This is a considerable value of the frequency noise, which has
direct impact in the phase noise. The same value was assessed with a longer
observational time (long term, typically 10 minutes). In Fig. 11(b), two
additional traces have been added to bound the maximum and minimum FM
variation. Here, the effect of the FM instability is more serious since the
RMS FM noise is 4.765 kHz (one order of magnitude higher than before). The
peak-to-peak noise was now 24 kHz (also one order of magnitude higher than at
short-time measurement).

Also, the IF output signal is FM demodulated and Fig. 11(a) shows the results
for the short term. As with the LO output signal, this noise was measured
during a short time window, typically less than two minutes observation time.
The RMS FM noise in this case was 426.3Hz and the peak-to-peak parameter is
2.5KHz. These noise levels are similar to the LO noise levels. The same
measurement with a longer time window is carried out, being the result shown
in Fig. 11(c). For a typical 10-minute time-window one has up to 4.5 KHz RMS
FM noise and 23KHz peak-to-peak FM noise. The RF output signal was also
monitored, at short term, with the results depicted on Fig. 11(a). Contrary to
LO and IF FM noise levels, RF exhibits less RMS frequency noise (27.39Hz) than
its predecessors. The amount of noise is one order of magnitude less than the
LO or IF noise. The window period for these measurements was less than two
minutes. The same measurement on the RF output level was performed for a
longer window period (around 10 minutes), as shown in Fig. 11(d). The RMS FM
noise in this case is 111.7 Hz and a peak-to-peak FM noise of 800 Hz
approximately.

Figure 11: (a) FM noise on LO, IF and RF signal (short term). (b) FM noise on
LO first product (long term). (c) FM noise on IF signal (long term). (d) FM
noise on RF signal (long term).

Another important measurement that can be compared with the OPTIMA testbed is
the SSB phase noise level on the output IF signal. Figure 12(a) shows the SSB
PN on the OPTIMA IF output signal whilst Fig. 12(b) shows the same SSB PN but
for the IF output signal in the OMO+PMFC testbed. The SSB phase noise is
higher in OMO+PMFC test bed than in OPTIMA testbed, especially at lower
frequency offsets. The main reason of this is that OMO is not optimized to
minimize phase noise. Apart from the dependence of the OM cavity on parameters
such as temperature, humidity and mechanical conditions, the optical path is
neither properly designed nor integrated to overcome all sources of phase
noise, like optical reflections, long fiber pigtails or use of single mode
fiber instead of PM fiber. However, this issue may be well solved by improving
the design of the cavity as well as its interface with the external system. In
this sense, a packaged device is mandatory for practical applications.

Figure 12: SSB phase noise of the IF signal. (a) OPTIMA PhLO; (b) OMO system.

##  VI Conclusions

In summary, we have characterized and discussed a new all-photonic microwave
oscillator, essentially performing as an integrated OEO but implemented using
an OM cavity built on a silicon chip, in a testbed suitable for validating
devices aimed at SATCOM applications. The OM performs as a photonic LO
operating approximately at multiples of the mechanical resonance frequency.
Looking at the performance as a OMO+PMFC downconverter, first-order down
conversion gain (RF-LO) exhibits results comparable to OPTIMA with equivalent
parameters (in this case, -32.8 dBm for both OPTIMA and OMO in configuration
3). This is a very positive aspect since it shows that the OMO can be
substitute of a PhLO, which requires of an external RF source, whilst the OMO
generates the LO in a purely optical way. Furthermore, higher-order down-
conversion gain (RF-n•LO) shows better results than the OPTIMA system with
equivalent parameters (-75.7 dB and -108 dB for OPTIMA RF-4LO and RF-5LO,
respectively). This is because the OMO design has a non-linearity factor
higher than the EOM-based photonic LO in OPTIMA or similar systems. This is a
remarkably good result as well, since higher LO frequencies can be generated
to get more frequency conversion bandwidth in a future system architecture.

One drawback is that the OMO optical sidebands exhibit less power than the
typical EOM-based PhLO configuration. This is because the mentioned non-
linearity factor in OMO. It gives more energy to the higher LO products, at
the expense of less power to the first LO sideband. This has a direct impact
in the system conversion gain, being lower in this solution than in the OPTIMA
system. About the signal integrity, another improvable two parameters are the
amplitude and the frequency stability. It has been seen that the output signal
products mixed with LO (n•LO and RF±n•LO) exhibit one order of magnitude more
FM noise than non-mixed products (particularly RF output signal). Also, the
amplitude noise is higher in the IF signal than in the RF output level.
Looking at other relevant parameters, the SSB phase noise is higher in this
assessed test board than in the OPTIMA test board.

From a practical perspective, the OMO can significantly reduce the SWaP (size,
weight and power) over the conventional approach due to its intrinsically
small footprint and the lack of control and biasing electronics. The device is
passive and can fit directly in conventional payload configurations (based on
Silicon-technology) without the need to implement specific interfaces. In
addition, it also provide inherent frequency mixing capabilities as a result
of the inherent nonlinear behavior of the OM cavity [24]. Therefore, with all
the advantages, this approach is a very promising candidate to build ultra-
light weight OEOs, highly appropriate for space applications.

## Acknowledgment

The authors acknowledge funding from the H2020 Future and Emerging
Technologies program (projects PHENOMEN 713450, SIOMO 945915 and OPTIMA
730149); the Spanish State Research Agency (PGC2018-094490-BC21 and
ICTS-2017-28-UPV-9); Generalitat Valenciana (BEST/2020/178, PROMETEO/2019/123,
IDIFEDER/2020/041 and IDIFEDER/2021/061).

## References

  * [1] M. A. Piqueras, J. Martí, S. Delgado, R. Singh, D. Beeder, R. Parish, G. Turgeon, A. Castells, and L. Roux, “A flight demonstration photonic payload for up to Q/V-band implemented in a satellite Ka-band hosted payload aimed at broadband high throughput satellites,” _International Conference on Space Optics_ , vol. 11180, 2019.
  * [2] M. A. Piqueras, J. Martí, S. Delgado, R. Singh, R. Parish, G. Turgeon, and A. Zornoza, “A Ka-band single string photonic payload flight demonstrator for broadband high throughput satellite systems and an in orbit demonstrator of optical RF distribution on board satellites,” _International Conference on Space Optics_ , vol. 11180, 2019. [Online]. Available: https://doi.org/10.1117/12.2536186
  * [3] V. Polo, “Microwave-photonics enabled flexible telecommunication payloads,” in _4th ESA Workshop in Advanced Flexible Payloads_ , 2019.
  * [4] OPTIMA, “Towards demonstration of photonic payload for telecom satellites, EU Project H2020-COMPET-2016-730149, https://www.photonic-payload-optima.eu/.”
  * [5] A. Bensky, _Short-range Wireless Communication_ , third edition ed. Newnes, 2019.
  * [6] X. S. Yao and L. Maleki, “Optoelectronic microwave oscillator,” _J. Opt. Soc. Am. B_ , vol. 13, no. 8, pp. 1725–1735, Aug 1996. [Online]. Available: http://josab.osa.org/abstract.cfm?URI=josab-13-8-1725
  * [7] M. Aspelmeyer, T. J. Kippenberg, and F. Marquardt, “Cavity optomechanics,” _Rev. Mod. Phys._ , vol. 86, pp. 1391–1452, 2014.
  * [8] L. Mercadé, L. L. Martín, A. Griol, D. Navarro-Urrios, and A. Martínez, “Microwave oscillator and frequency comb in a silicon optomechanical cavity with a full phononic bandgap,” _Nanophotonics_ , vol. 9, no. 11, pp. 3535–3544, 2020.
  * [9] Y. Pennec, V. Laude, N. Papanikolaou, B. Djafari-Rouhani, M. Oudich, S. E. Jallal, J. C. Beugnot, J. M. Escalante, and A. Martínez, “Modeling light-sound interaction in nanoscale cavities and waveguides,” _Nanophotonics_ , vol. 3, no. 6, pp. 413 – 440, 2014.
  * [10] T. J. Kippenberg and K. J. Vahala, “Cavity optomechanics: Back-action at the mesoscale,” _Science_ , vol. 321, no. 5893, pp. 1172–1176, 2008.
  * [11] D. Navarro-Urrios, J. Gomis-Bresco, S. El-Jallal, M. Oudich, A. Pitanti, N. Capuj, A. Tredicucci, F. Alzina, A. Griol, Y. Pennec, B. Djafari-Rouhani, A. Martínez, and C. M. Sotomayor Torres, “Dynamical back-action at 5.5 ghz in a corrugated optomechanical beam,” _AIP Advances_ , vol. 4, no. 12, p. 124601, 2014.
  * [12] M. Hossein-Zadeh and K. J. Vahala, “An optomechanical oscillator on a silicon chip,” _IEEE Journal of Selected Topics in Quantum Electronics_ , vol. 16, no. 1, pp. 276–287, Jan 2010.
  * [13] I. Ghorbel, F. Swiadek, R. Zhu, D. Dolfi, G. Lehoucq, A. Martin, G. Moille, L. Morvan, R. Braive, S. Combrié, and A. De Rossi, “Optomechanical gigahertz oscillator made of a two photon absorption free piezoelectric iii/v semiconductor,” _APL Photonics_ , vol. 4, no. 11, p. 116103, 2019.
  * [14] F. Liu and M. Hossein-Zadeh, “Characterization of optomechanical rf frequency mixing/down-conversion and its application in photonic rf receivers,” _Journal of Lightwave Technology_ , vol. 32, no. 2, pp. 309–317, 2014.
  * [15] K. Huang and M. Hossein-Zadeh, “Underwater acoustic signal detection and down-conversion using optomechanical resonance and oscillation,” _Journal of Lightwave Technology_ , vol. 38, no. 14, pp. 3789–3797, 2020\.
  * [16] M. Eichenfield, J. Chan, R. M. Camacho, K. J. Vahala, and O. Painter, “Optomechanical crystals,” _Nature_ , vol. 462, no. 7269, pp. 78–82, 2009\.
  * [17] J. Gomis-Bresco, D. Navarro-Urrios, M. Oudich, S. El-Jallal, A. Griol, D. Puerto, E. Chavez, Y. Pennec, B. Djafari-Rouhani, F. Alzina, A. Martínez, and C. M. S. Torres, “A one-dimensional optomechanical crystal with a complete phononic band gap,” _Nature Communications_ , vol. 5, no. 1, p. 4452, 2014.
  * [18] D. Navarro-Urrios, N. E. Capuj, J. Gomis-Bresco, F. Alzina, A. Pitanti, A. Griol, A. Martínez, and C. M. Sotomayor Torres, “A self-stabilized coherent phonon source driven by optical forces,” _Scientific Reports_ , vol. 5, p. 15733, 2015.
  * [19] L. Mercadé, K. Pelka, R. Burgwal, A. Xuereb, A. Martínez, and E. Verhagen, “Floquet phonon lasing in multimode optomechanical systems,” _Phys. Rev. Lett._ , vol. 127, p. 073601, 2021.
  * [20] H. Ren, M. H. Matheny, G. S. MacCabe, J. Luo, H. Pfeifer, M. Mirhosseini, and O. Painter, “Two-dimensional optomechanical crystal cavity with high quantum cooperativity,” _Nature Communications_ , vol. 11, no. 1, p. 3373, 2020.
  * [21] M. Oudich, S. El-Jallal, Y. Pennec, B. Djafari-Rouhani, J. Gomis-Bresco, D. Navarro-Urrios, C. M. Sotomayor Torres, A. Martínez, and A. Makhoute, “Optomechanic interaction in a corrugated phoxonic nanobeam cavity,” _Phys. Rev. B_ , vol. 89, p. 245122, 2014.
  * [22] J. Chan, “Laser cooling of an optomechanical crystal resonator to its quantum ground state of motion,” Ph.D. dissertation, California Institute of Technology, 2012.
  * [23] J. Chan, A. H. Safavi-Naeini, J. T. Hill, S. Meenehan, and O. Painter, “Optimized optomechanical crystal cavity with acoustic radiation shield,” _Applied Physics Letters_ , vol. 101, no. 8, p. 081115, 2012.
  * [24] L. Mercadé, M. Morant, A. Griol, R. Llorente, and A. Martínez, “Photonic frequency conversion of OFDM microwave signals in a wavelength-scale optomechanical cavity,” _Laser & Photonics Reviews_, vol. 15, no. 11, p. 2100175.
  * [25] E. Rubiola, _Phase Noise and Frequency Stability in Oscillators_ , ser. The Cambridge RF and Microwave Engineering Series. Cambridge University Press, 2008.
  * [26] X. Luan, Y. Huang, Y. Li, J. F. McMillan, J. Zheng, S.-W. Huang, P.-C. Hsieh, T. Gu, D. Wang, A. Hati, D. A. Howe, G. Wen, M. Yu, G. Lo, D.-L. Kwong, and C. W. Wong, “An integrated low phase noise radiation-pressure-driven optomechanical oscillator chipset,” _Scientific Reports_ , vol. 4, p. 6842, Oct. 2014. [Online]. Available: https://doi.org/10.1038/srep06842
  * [27] W. Yu, W. C. Jiang, Q. Lin, and T. Lu, “Cavity optomechanical spring sensing of single molecules,” _Nature Communications_ , vol. 7, no. 1, p. 12311, 2016\.
  * [28] N. Karafolas, J. M. P. Armengol, and I. Mckenzie, “Introducing photonics in spacecraft engineering: Esa’s strategic approach,” in _2009 IEEE Aerospace conference_ , 2009, pp. 1–15.