# The North System for Formosa Speech Recognition Challenge 2023

Li-Wei Chen 陳力瑋
North Co., Ltd., Taiwan
Kai-Chen Cheng 鄭楷蓁
North Co., Ltd., Taiwan
Hung-Shin Lee 李鴻欣
North Co., Ltd., Taiwan

###### Abstract

This report provides a concise overview of the proposed North system, which
aims to achieve automatic word/syllable recognition for Taiwanese Hakka
(Sixian). The report outlines three key components of the system: the
acquisition, composition, and utilization of the training data; the
architecture of the model; and the hardware specifications and operational
statistics. The demonstration of the system has been made
public111https://shorturl.at/mzGL7.

Keywords:  Hakka, Sixian, speech recognition

##  1 Introduction

This document furnishes a succinct yet comprehensive overview of the proposed
North System, a technologically advanced mechanism designed with the primary
objective of achieving automatic recognition of words and syllables specific
to the Taiwanese Hakka language, with a particular focus on the Sixian
dialect. The report meticulously delineates three pivotal components integral
to the effective functionality and operation of the system, as enumerated
below:

  1. 1.

Acquisition, Composition, and Utilization of Training Data:

     * •

Acquisition: The systematic collection and sourcing of relevant linguistic
data pertinent to the Taiwanese Hakka language, ensuring a robust and
representative dataset.

     * •

Composition: The strategic assembly and organization of the acquired data,
ensuring it is structured in a manner conducive to effective machine learning.

     * •

Utilization: The application of the composed data in training the system,
ensuring it accurately and efficiently recognizes and processes linguistic
elements of the Sixian dialect.

  2. 2.

Architectural Framework of the Model:

     * •

A detailed exposition of the structural and operational framework of the
model, elucidating the technological and algorithmic methodologies employed to
facilitate accurate linguistic recognition and processing.

     * •

An exploration of the model’s design principles, underlying algorithms, and
computational processes that enable it to effectively learn, recognize, and
interpret the linguistic nuances of the Taiwanese Hakka language.

  3. 3.

Hardware Specifications and Operational Metrics:

     * •

Hardware Specifications: A thorough breakdown of the technological
infrastructure supporting the system, detailing the hardware components and
specifications that underpin its operation.

     * •

Operational Metrics: An analytical overview of the system’s performance
metrics, providing insights into its operational efficiency, accuracy, and
reliability in real-world applications.

##  2 Data

Table 1 meticulously presents a detailed inventory of our principal corpus
sources, accompanied by their respective statistical details, providing an in-
depth insight into the voluminous data utilized in our research and
development endeavors.

In addition to the primary corpus, our research team has assiduously gathered
an extensive collection of Hakka (Sixian dialect)-related speech data,
exceeding 100 hours, from a variety of online platforms. This includes, but is
not limited to:

  * •

YouTube: A prominent video-sharing platform where a myriad of Hakka (Sixian)
linguistic content, ranging from casual conversations to formal discourses,
has been extracted.

  * •

Podcasts: Audio programs and series that provide a rich source of
conversational and narrative Hakka (Sixian) speech data.

  * •

Additional Online Platforms: Various other digital platforms that host a
wealth of linguistic content pertinent to the Hakka (Sixian) dialect.

Moreover, we have amassed a substantial volume of Hakka text data,
specifically curated for language modeling purposes, from a plethora of
websites ardently dedicated to the promotion, preservation, and dissemination
of the Hakka language and its diverse dialects.

Our methodologies for obtaining speech data are not solely confined to direct
data gathering but also partially derive inspiration from the scholarly paper
presented at O-COCOSDA 2020 by Dr. Hung-Shin Lee Chen et al. (2020). This
paper provides valuable insights and methodologies that have been judiciously
considered and adapted to enhance our data acquisition strategies.

It is imperative to underscore that our research and development team
consciously opted to abstain from employing speech synthesis as a means for
generating training speech data. This decision is firmly rooted in our belief
that, while speech synthesis may offer a modicum of utility in certain
contexts, its efficacy is notably constrained in the realm of speech
recognition due to the intrinsic emphasis on accommodating and understanding
variability in speaker characteristics and environmental acoustics.

Table 1: Statistics of sources of Hakka (Sixian) speech. The speech channels
used in Official (train) and Official (pilot-test) are lavalier and Zoom,
respectively. Hakka Dictionary comes from Dictionary of Frequently-Used Taiwan
Hakka (https://hakkadict.moe.edu.tw). HAC, provided by Hakka Affairs Council
(https://corpus.hakka.gov.tw), is further cleaned by our technology. SPU
denotes average seconds per utterance.

Source | Hours | # Utt. | SPU
---|---|---|---
Official Dataset (train) | 59.43 | 20,591 | 10.39
Official Dataset (pilot-test) | 10.01 | 3,595 | 10.02
Hakka Dictionary | 5.84 | 15,250 | 1.38
HAC | 11.26 | 4,216 | 9.61
Total | 86.54 | 43,652 | 7.14

##  3 Model Structure

The meticulous training of the acoustic model necessitates the concatenation
of two distinct types of speech features: the 40-dimensional Mel Frequency
Cepstral Coefficients (MFCCs) and the 1024-dimensional Semi-Supervised
Learning (SSL) embeddings. The SSL model, which was previously subjected to
training on a comprehensive dataset of Chinese linguistic data utilizing the
HuBERT-large model, is judiciously utilized to procure the SSL embeddings.

The model employed is fundamentally based on two primary architectural
structures. Firstly, the Chain-based Discriminative Autoencoder (DcAE) Lee et
al. (2022), and secondly, the Multistream Convolutional Neural Network (CNN)
Han et al. (2021). For an in-depth elaboration and comprehensive understanding
of these structures, readers are ardently encouraged to consult the referenced
scholarly papers or the Appendix at the end of the paper. The integration of
both aforementioned structures is employed in a joint training methodology,
wherein the latter structure serves as the foundational bedrock upon which the
former is developed and refined. The overarching objective permeating the
entirety of the model is to assiduously minimize losses associated with
Automatic Speech Recognition (ASR), particularly those pertaining to lattice-
free Maximum Mutual Information (MMI), as well as errors inherent in feature
reconstruction and restoration. Through a meticulous analysis of speech and
noise factors, and the incorporation of multi-resolution information, the
model significantly enhances its performance and robustness in various
linguistic contexts.

The rescoring mechanism, which is meticulously designed to operate on the word
lattice generated by a four-gram language model, employs a Recurrent Neural
Network Language Model (RNN-LM) to enhance the accuracy and reliability of
linguistic predictions and outputs.

It is pivotal to note that, consequent to the insufficiency of Graphical
Processing Unit (GPU) resources, our team made a conscious decision to abstain
from proceeding with the optimization and fine-tuning of model parameters and
hyperparameters. This includes, but is not limited to, the number of model
layers, training epochs, batch size, learning rate, among other crucial
variables that significantly impact the model’s learning and predictive
capabilities.

##  4 Hardware

In the execution of our experiment, we strategically employed a total of four
NVIDIA RTX™ A5000 graphics cards, which are renowned for their robust
computational capabilities and adeptness in handling graphically-intensive
tasks and processes. It is imperative to underscore that this allocation of
graphical processing units (GPUs) was the sole resource utilized to facilitate
the extensive computational demands inherent in the training of the neural
model.

The entirety of the training duration for a singular neural model was
approximately 82 hours, a substantial temporal investment that underscores the
computational and temporal demands associated with the development and
refinement of sophisticated neural networks. This extensive training period
was meticulously undertaken to ensure the model was afforded ample opportunity
to learn, adapt, and refine its predictive capabilities, thereby enhancing its
overall performance and reliability in practical applications.

##  5 Final Results

The conclusive results, which encapsulate the outcomes and findings derived
from our experimental processes, are meticulously presented in Table 2. Our
North system got the double champion of this Formosa Speech Recognition
Challenge 2023 (General Group) in the two tracks of Hakka Character and
Pinyin. It is imperative to underscore that one of the predominant weaknesses
that permeated our model during the training phase is the conspicuous absence
of spontaneous speech. This deficiency in the training data potentially
impacts the model’s capacity to accurately and reliably recognize and process
unscripted, natural linguistic patterns and variations, thereby presenting an
area warranting further investigation and enhancement in future research
endeavors.

We also compared the Hakka ASR system developed by
ASUS222https://www.asuscloud.com/event-hakka, which is built up based on
OpenAI’s Whisper. Table 3 shows that our system is far superior to ASUS. It is
worth noting that ASUS’s system cannot handle sentences that are too long,
meaning that 384 utterances could not be processed and were not included in
the error rate calculation.

Table 2: Final results of North ASRs with respect to two tracks, 客語漢字 (Hakka Character) and 客語拼音 (Hakka Pinyin), in terms of character and syllable error rate (%), respectively. The proportion of reading and spontaneous speech in the total evaluation data is approximately 13% and 87% respectively. The total number of utterances for evaluation is 5,913. Track | Read | Spont. | Average | Rank
---|---|---|---|---
客語漢字 | 4.27 | 19.14 | 17.15 | 1
客語拼音 | 7.33 | 18.90 | 17.42 | 1
Table 3: Comparisons of North ASRs and ASUS Hakka ASRs respect to two tracks, 客語漢字 (Hakka Character) and 客語拼音 (Hakka Pinyin), in terms of character and syllable error rate (%), respectively. Because the ASUS Hakka ASR cannot deal with long utterances, only 5,529 utterances are evaluated. Track | ASUS | North | Rel. Improve.
---|---|---|---
客語漢字 | 28.87 | 18.17 | 37.06
客語拼音 | 42.43 | 19.65 | 53.69

## Acknowledgments

Regarding the collection and understanding of Hakka language corpus, we would
like to express our gratitude to the following individuals for their
professional assistance:

  * •

黃麗芬（北一女中客語教師）

  * •

徐煥昇（育英國小龍騰分校教師）

  * •

張陳基（國立聯合大學文化創意與數位行銷學系教授）

## Appendix

### Multistream CNN Han et al. (2021)

The Multistream Convolutional Neural Network (CNN), a pioneering neural
network architecture, meticulously designed to fortify acoustic modeling
within the realm of speech recognition tasks, is introduced. The architectural
framework processes input speech utilizing diverse temporal resolutions,
achieved by applying varying dilation rates to convolutional neural networks
across multiple streams, thereby attaining robustness in acoustic modeling.
The dilation rates are judiciously selected from multiples of a sub-sampling
rate, specifically, three frames. Each stream systematically stacks Time Delay
Neural Network-F (TDNN-F) layers, a variant of 1D CNN, and the output
embedding vectors derived from the streams are concatenated and subsequently
projected to the terminal layer.

The efficacy of the Multistream CNN architecture is validated through
demonstrable and consistent enhancements against Kaldi’s optimal TDNN-F model,
observed across a myriad of data sets. The Multistream CNN facilitates a 12%
(relative) improvement in the Word Error Rate (WER) of the test-other set
within the LibriSpeech corpus. Furthermore, on custom data derived from
ASAPP’s production Automatic Speech Recognition (ASR) system for a contact
center, it records a relative WER enhancement of 11% for customer channel
audio, thereby substantiating its robustness to data in uncontrolled
environments. In terms of the real-time factor, the Multistream CNN surpasses
the baseline TDNN-F by 15%, thereby also indicating its practical
applicability within production systems. When amalgamated with self-attentive
Simple Recurrent Unit (SRU) Language Model (LM) rescoring, the Multistream CNN
significantly contributes to ASAPP achieving an optimal WER of 1.75% on the
test-clean set in LibriSpeech.

### Chain-based Discriminative Autoencoder Lee et al. (2022)

In preceding research endeavors, the authors introduced a model known as the
Discriminative Autoencoder (DcAE), specifically tailored for applications
within the domain of speech recognition. The DcAE amalgamates two distinct
training schemes into a singular, cohesive model. Initially, as the DcAE is
designed with the objective of learning encoder-decoder mappings, it endeavors
to minimize the squared error between the reconstructed speech and the
original input speech. Subsequently, within the code layer, frame-based
phonetic embeddings are procured by minimizing the categorical cross-entropy
between the ground truth labels and the predicted triphone-state scores. The
development of DcAE is grounded in the Kaldi toolkit, wherein various Time
Delay Neural Network (TDNN) models are treated as encoders.

In the Chain-based DcAE, they further introduce three novel iterations of the
DcAE. Firstly, a new objective function is employed, which takes into
consideration both the categorical cross-entropy and the mutual information
between ground truth and predicted triphone-state sequences, resulting in the
formulation of a chain-based DcAE (c-DcAE). To facilitate its application to
robust speech recognition, they further extend c-DcAE to incorporate
hierarchical and parallel structures, culminating in the development of hc-
DcAE and pc-DcAE, respectively. Within these two models, both the error
between the reconstructed noisy speech and the input noisy speech, as well as
the error between the enhanced speech and the reference clean speech, are
integrated into the objective function.

Experimental results, derived from the Wall Street Journal (WSJ) and Aurora-4
corpora, substantiate that the DcAE models exhibit superior performance when
juxtaposed with baseline systems, thereby affirming their efficacy in speech
recognition tasks.

## References

  * Chen et al. (2020) Pin-Yuan Chen, Chia-Hua Wu, Hung-Shin Lee, Shao-Kang Tsao, Min-Tak Ko, and Hsin-Min Wang. 2020.  Using taigi dramas with mandarin chinese subtitles to improve taigi speech recognition.  In _Proc. O-COCOSDA_.
  * Han et al. (2021) Kyu Han, Jing Pan, Venkata Krishna, Naveen Tadala, Tao Ma, and Dan Povey. 2021.  Multistream CNN for robust acoustic modeling.  In _Proc. ICASSP_.
  * Lee et al. (2022) Hung-Shin Lee, Pin-Tuan Huang, Yao-Fei Cheng, and Hsin-Min Wang. 2022.  Chain-based discriminative autoencoders for speech recognition.  In _Proc. Interspeech_.


RBI-ThPhys-2022-19

# A short overview on low mass scalars
at future lepton colliders

Tania Robens<EMAIL_ADDRESS>Ruder Boskovic Institute, Bijenicka cesta 54,
10000 Zagreb, Croatia  Theoretical Physics Department, CERN, 1211 Geneva 23,
Switzerland 111Previous affiliation.

######  Abstract

In this manuscript, I give a short summary on scenarios with new physics
scalars that could be investigated at future $e^{+}e^{-}$ colliders. I
concentrate on cases where at least one of the additional scalar has a mass
below 125 GeV, and discuss both models where this could be realized, as well
as studies which focus on such scenarios. This work is based on several
overview talks I recently gave at the CEPC workshop, FCC week and ECFA future
collider workshop, as well as a Snowmass White Paper.

##  I Introduction

The discovery of a scalar which so far largely agrees with predictions for the
Higgs boson of the Standard Model (SM) has by now been established by the LHC
experiments (see e.g. atlaspub; cmspub), with analyses of Run II LHC data
further confirming this. In the European Strategy Report
EuropeanStrategyforParticlePhysicsPreparatoryGroup:2019qin; European:2720129,
a large focus was put on future $e^{+}e^{-}$ colliders, especially so-called
Higgs factories with center-of-mass (com) energies around $240\,-250\,{\rm
GeV}$. While these will on the one hand further help to determine properties
of the scalar discovered at the LHC, and especially will help to determine in
detail the parameters and shape of the scalar potential, it is also
interesting to investigate their potential to search for additional scalar
states. In general, such states can be realized by extending the scalar sector
of the SM by additional gauge singlets, doublets, triplets, or other
multiplets. Depending on the specific extension, these models then contain
several additional neutral or (multiply) charged scalar states. I here
concentrate on scenarios where at least one of these has a mass
$\lesssim\,125\,{\rm GeV}$.

In this work I give a short overview on some models that allow for such light
states, and point to phenomenological studies investigating such models. This
should be viewed as an encouragement for further detailed studies in this
direction. A preliminary version of this overview has been submitted as a
Snowmass White Paper Robens:2022erq.

##  II Models

###  II.1 Singlet extensions

In singlet extensions (see e.g. Pruna:2013bma; Robens:2015gla; Robens:2016xkb;
Ilnicka:2018def; Robens:2019kga), the SM scalar potential is enhanced by
additional scalar states that are singlets under the SM gauge group. In such
scenarios, the coupling of the novel scalar(s) to SM particles is typically
inherited via mixing, i.e. mass-eigenstates are related to gauge eigenstates
via a unitary mixing matrix. The corresponding couplings and interactions are
mediated via a simple mixing angle. In general, in such models scalars have
decay modes that connect them to both the SM as well as the novel scalar
sector. If the additional states are singlets, all interactions to SM final
states are rescaled by a mixing angle, such that

$g_{h_{i},\text{SM}\text{SM}}\,=\,\sin\alpha_{i}\,\times\,g^{\text{SM}}_{h_{i},\text{SM}\text{SM}}$

for general SMSM final states, where $h_{i}$ and $\alpha_{i}$ denote the
respective scalar and mixing angle. Depending on the specific kinematics, also
novel triple and quartic interactions are possible, where the couplings depend
on the specific values of the potential parameters. This allows in general for
decays $h_{i}\,\rightarrow\,h_{j}\,h_{k}$. Naturally, one of the CP even
neutral scalars of the model has to have properties that concur with current
measurements by the LHC experiments.

In Cepeda:2021rql, the authors present the status of current searches for the
process

$p\,p\,\rightarrow\,h_{125}\,\rightarrow\,h_{i}\,h_{i}\,\rightarrow\,X\,X\,Y\,Y$
(1)

which for such models can be read as a bound in

$\sin^{2}\,\alpha\,\times\,\text{BR}_{h_{125}\,\rightarrow\,h_{i}\,h_{i}\,\rightarrow\,X\,X\,Y\,Y}.$

We display these results in figure 1. Current bounds on the mixing angle for
the 125 GeV-like state are around $|\sin\,\alpha|\,\lesssim\,0.3$
Robens:2022zav, which in turn means that branching ratios
$\text{BR}_{h_{125}\,\rightarrow\,h_{i}\,h_{i}\,\rightarrow\,X\,X\,Y\,Y}$ down
to $\mathcal{O}\left(10^{-5}\right)$ can be tested. In particular the
$\mu\mu\mu\mu$ final states in the low mass region give interesting
constraints on the $h_{125}\,\rightarrow\,h_{i}\,h_{i}$ branching ratio down
to $\,\sim\,0.03$.

Figure 1:  Limits on the process in eqn (1), taken from Cepeda:2021rql. This
displays current constraints which can especially be easily reinterpreted in
extended scalar sector models, in particular models where couplings are
inherited via a simple mixing angle. In this figure, the lighter scalar is
denoted by $s$, which corresponds to $h_{i}$ in the notation used in this
manuscript.

We also refer the reader to a related discussion in Carena:2022yvx.

After this short overview on current constraints, we now show an example of
the allowed parameter space in a model with two additional singlets, the two
real scalar extension studied in Robens:2019kga. In this model, three CP-even
neutral scalars exist that relate the gauge and mass eigenstates $h_{1,2,3}$
via mixing. One of these states has to have couplings and mass complying with
current measurements of the SM-like scalar, the other two can have higher or
lower masses. A detailed discussion of the model including theoretical and
experimental constraints, can be found in Robens:2019kga. In figure 2, we
display two cases where either one (high-low) or two (low-low) scalar masses
are smaller than $125\,{\rm GeV}$. On the y-axis, the respective mixing angle
is shown. Complete decoupling would be designated by $\sin\alpha\,=\,0$ in the
notation used in this figure.

Figure 2:  Available parameter space in the TRSM, with one (high-low) or two
(low-low) masses lighter than 125 GeV. Left: light scalar mass and mixing
angle, with $\sin\alpha\,=\,0$ corresponding to complete decoupling. Right:
available parameter space in the $\left(m_{h_{1}},\,m_{h_{2}}\right)$ plane,
with color coding denoting the rescaling parameter $\sin\alpha$ for the
lighter scalar $h_{1}$. Within the green triangle,
$h_{125}\,\rightarrow\,h_{2}h_{1}\,\rightarrow\,h_{1}\,h_{1}\,h_{1}$ decays
are kinematically allowed.

The points were generated using ScannerS Coimbra:2013qq; Muhlleitner:2020wwk,
interfaced to HiggsBounds-5.10.2 Bechtle:2008jh; Bechtle:2011sb;
Bechtle:2013wla; Bechtle:2020pkv and HiggsSignals-2.6.2 Bechtle:2013xfa;
Bechtle:2020uwn, with constraints as implemented in these versions. Dominant
decays are inherited from SM-like scalars with the respective masses,
rendering the $b\bar{b}b\bar{b}$ final state dominant for large regions of
this parameter space. Note that in the case where both additional scalars have
masses $\lesssim\,125\,{\rm GeV}$, $h_{1}\,h_{1}\,h_{1}$ final states would
also be allowed in certain regions of parameter space. Note that the points
additionally fulfill the novel upper bound on the total width of the 125 GeV
scalar of 5.6 MeV CMS:2022ley.

###  II.2 Two Higgs Doublet Models

Two Higgs doublet models (2HDMs) constitute another example of new physics
models allowing for low mass scalar states. A general discussion of such
models is e.g. given in Branco:2011iw and will not be repeated here. In
general, such models contain, besides the SM candidate, two additional neutral
scalars which differ in CP properties as well as a charged scalar, so the
particle content is given by $h,\,H,\,A,\,H^{\pm}$, where one of the two CP-
even neutral scalars $h,\,H$ needs to be identified with the 125 GeV resonance
discovered at the LHC. Couplings to the fermions in the Yukawa sector
distinguish different types of 2HDMs.

In Eberhardt:2020dat, the authors perform a scan including bounds from theory,
experimental searches and constraints, as e.g. electroweak observables, as
well as B-physics. Examples for these scan results are shown in figure 3,
taken from that reference.

Figure 3:  Allowed regions in the 2HDM, from a scan presented in
Eberhardt:2020dat.

We see that for all regions solutions for either one or several low mass
scenarios exist and are viable for the constraints discussed in that
reference. The authors here primarily consider an aligned 2HDM Pich:2009sp;
Pich:2010ic, including an angle $\tilde{\alpha}$ that parametrizes the
rescaling with respect to the Standard Model couplings to gauge bosons, with
$\cos\tilde{\alpha}\,=\,0$ designating the SM decoupling.

The limits on the absolute value of the cosine of rescaling angle vary between
0.05 and 0.25 ATLAS-CONF-2021-053. In figure 4, we display this angle vs the
different scalar masses, reproduced from Eberhardt:2020dat222I thank V.
Miralles for providing these plots.. We see that all regions for masses
$\lesssim\,125\,{\rm GeV}$ can be populated, with absolute value of mixing
angle ranges $|\cos\left(\tilde{\alpha}\right)|\lesssim\,0.1$.

Figure 4:  Mixing angle and masses of different additional scalars in the
aligned 2HDM, from the scan presented in Eberhardt:2020dat. For all additional
scalars, regions exists where masses are $\lesssim\,125\,{\rm GeV}$, with
absolute values of mixing angles such that
$|\cos\left(\tilde{\alpha}\right)|\lesssim\,0.1$.

Other examples for 2HDMs including possibilities for low-mass scalars, in
particular for the CP-odd candidate $A$, can be found in Wang:2022yhm.

###  II.3 Other extensions

The scalar sector of the SM can be extended by an arbitrary number of
additional scalar fields, such as singlets, doublets, etc. One option which is
also often considered is the extension of this sector by both singlets and
doublets. We will discuss a couple of examples of such extensions which are
able to provide regions where some of the additional scalars are in the low-
mass range of interest here. For more details on the specific models, we refer
the readers to the respective publications.

####  II.3.1 N2HDM

In Abouabid:2021yvw, the authors consider a model where the SM scalar sector
is extended by an additional doublet as well as a real singlet. This model has
3 CP even neutral scalar particles, out of which one needs to have the
properties in compliance with LHC measurements of the 125 GeV scalar. The
authors perform an extensive scan and find regions in parameter space where
either one or both of the additional scalars have masses below 125 GeV. We
show an example of the allowed parameter space in figure 5. We see that in the
CP-even sector there are regions within this model that still allow for low
mass scalars.

Figure 5:  Scan results in the N2HDM, taken from Abouabid:2021yvw. There are
regions in the models parameter space where either one or two of the
additional scalars have masses $\lesssim\,125\,{\rm GeV}$.

####  II.3.2 Lepton-specific Inert Doublet Model

In Han:2021gfu, the authors consider a model where the SM scalar sector is
augmented by an additional doublet, where they impose an exact
$\mathbb{Z}_{2}$ symmetry. This symmetry is then broken by a specific coupling
to the fermionic sector. The authors identify regions in the models parameter
space that agree with current searches as well as anomalous magnetic momenta
of electron and muon, and find regions where the second CP-even scalar can
have a mass $\lesssim\,30\,{\rm GeV}$. We display these regions in figure 6.

Figure 6:  Allowed regions in the parameter space of the model discussed in
Han:2021gfu, taken from that reference, where squares denote allowed and
bullets excluded regions in the models parameter space. CP-even neutral
scalars with low masses are viable within this model.

####  II.3.3 Scalar triplet model

Finally, we want to discuss a model containing scalar triplets, leading to a
rich particle content as well as the possibility of CP violating terms. The
model has been presented in Ferreira:2021bdj. This model contains 5 neutral, 3
charged, and 2 doubly charged mass eigenstates. The authors present regions in
parameter space where masses for some of these can be $\lesssim\,125\,{\rm
GeV}$. We display these results in figure 7. We see that both neutral and
charged scalars with low mass realizations exist, with a rescaling of the
$hZZ$ coupling below LEP sensitivity. These scenarios could be investigated at
future machines with higher sensitivity in this region.

Figure 7:  Allowed regions in the parameter space of the model discussed in
Ferreira:2021bdj, taken from that reference. For neutral and charged new
scalars, masses $\lesssim\,125\,{\rm GeV}$ are achievable.

##  III Studies at 90 GeV

For this center-of-mass energy, several searches exist which have already been
performed at LEP and are summarized in OPAL:2002ifx; ALEPH:2006tnd,
concentrating on $Z\,h$, $h_{1}\,h_{2},$ and $h_{1}\,h_{1}\,h_{1}$ final
states, where $h_{i}$ signifies novel scalars. Possible new studies could
build upon these searches. We want to note that the luminosity at FCC-ee and
CEPC at this center of mass energy is exceeding LEP luminosity by several
orders of magnitude FCC:2018evy; CEPCStudyGroup:2018ghi.

We also want to draw attention to one specific study which investigates
several composite models at a com energy of 91 GeV Cornell:2020usb. The
authors consider the process
$e^{+}e^{-}\,\rightarrow\,\ell^{+}\ell^{-}\tau^{+}\tau^{-}$, where the tau-
pair stems from an additional pseudoscalar $a$ radiated off one of the fermion
lines in the $\ell\ell$ pair-production. They apply a cut-based study as well
as an improved analysis using machine learning techniques; for the latter, the
authors are able to achieve a 3 $\sigma$ exclusion for benchmarks with masses
$M_{a}\,\sim\,20\,{\rm GeV}$. We display event rates for the various benchmark
scenarios in figure 8.

Figure 8:  Rates at a 91 GeV FCC-ee for various models discussed in
Cornell:2020usb, for $\ell^{+}\ell^{-}\tau^{+}\tau^{-}$ final states. M7 and
M10 reach a 3 $\sigma$ significance using ML techniques. Figure taken from
Cornell:2020usb.

##  IV Studies at 240-250 GeV

Throughout this work, we show for reference leading order predictions for $Zh$
production at $e^{+}e^{-}$ colliders for low mass scalars which are SM-like.
These results were obtained using Madgraph5 Alwall:2011uj and are given for
approximate reference only. We also display the VBF-type production of
$e^{+}e^{-}\,\rightarrow\,h\,\nu_{\ell}\,\bar{\nu}_{\ell}$. Note that the
latter signature also contains contributions from $Z\,h$ production, where
$Z\,\rightarrow\,\nu_{\ell}\,\bar{\nu}_{\ell}$.

Figure 9 shows the production cross sections for these processes for a center-
of-mass energy of $\,\sim\,240\,-\,250\,{\rm GeV}$. Using these predictions,
and taking into account standard rescaling $\sim\,0.1$, around $10^{5}-10^{6}$
events could be produced with ILC, FCC-ee, and CEPC design luminosities
Bambade:2019fyw; FCC:2018evy; CEPCStudyGroup:2018ghi.

Figure 9:  Leading order production cross sections for $Z\,h$ and
$h\,\nu_{\ell}\,\bar{\nu}_{\ell}$ production at an $e^{+}\,e^{-}$ collider
with a com energy of 240 GeV(left) and 250 GeV (right) using Madgraph5 for an
SM-like scalar h. Shown is also the contribution of $Z\,h$ to
$\nu_{\ell}\,\bar{\nu}_{\ell}\,h$ using a factorized approach for the Z decay.

###  IV.1 Dedicated studies

####  IV.1.1 Light scalars in $Zh$ production

Not many dedicated studies exist that investigate low-mass scalars at Higgs
factories. We here point to Drechsel:2018mgd that investigates the sensitivity
of the ILC for low-mass scalars in $Z\,h$ production, either using pure $Z$
recoil (”recoil method”) or taking the light scalar decay into $b\,\bar{b}$
into account. The y-axis shows the $95\,\%$ CL limit for agreement with a
background only hypothesis, which can directly be translated into an upper
bound on rescaling. The authors validate their method by reproducing LEP
results LEPWorkingGroupforHiggsbosonsearches:2003ing; ALEPH:2006tnd for these
channels prior to applying their method to the ILC. Their predictions are
shown in figure 10.

Figure 10:  Sensitivity predictions for an ILC with a com energy of 250 GeV
from Drechsel:2018mgd. See text for details.

A more detailed study along similar lines using the recoil method only and
comparing different detector options has been presented recently in
Wang:2020lkq. We display their results in figure 11. The authors perform their
analysis in a model where the coupling of the new resonance is rescaled by a
mixing angle $\sin\theta$; therefore, their results can directly be compared
with the ones presented in Drechsel:2018mgd and figure 10.

Figure 11:  Upper bounds on the mixing angle for the model discussed in
Wang:2020lkq, in a comparison of different detector concepts and using the
recoil method.

####  IV.1.2 Higgsstrahlung and decay into two light scalars

In Liu:2016zki, the authors consider Higgs-strahlung at a 240 GeV $e^{+}e^{-}$
collider, where the Higgs subsequently decays into two light scalar states.
They give 95 $\%$ confidence level bounds for the branching ratios into the
decay products of the two light scalars as a function of the light scalar
masses for an integrated luminosity of $\int\mathcal{L}\,=\,5\,{\rm ab}^{-1}$
following a detailed study. Their results are subsequently used by many
authors as standard reference. We show their results for various channels in
figure 12.

Figure 12:  95 $\%$ confidence bounds on branching ratios for Higgs decay into
a pair of lighter particles, for a com energy of 240 GeVand
$\int\mathcal{L}\,=\,5\,{\rm ab}^{-1}$. Taken from Liu:2016zki.

Depending on the mass, model, and decay mode, branching ratios down to
$\sim\,10^{-4}$ can be tested.

A more recent study Shelton:2021xwo investigates the same process, but for
$4\,\tau$ final states, for the same center-of-mass energy and integrated
luminosity. The results, for varying values of tracking efficiency, are shown
in figure 13. Note that curent constraints on the invisible branching ratio of
the Higgs, the signal strength, as well as SM-like decays of the light scalars
currently render a bound $\lesssim\,10^{-2}$.

Figure 13:  Bounds on decay of the SM Higgs boson into two light scalars, with
a 4 $\tau$ final state, at an $e^{+}e^{-}$ collider with a com energy of 240
GeV, with different assumptions on tracking efficiencies. Taken from
Shelton:2021xwo.

Several works make use of the bounds derived in Liu:2016zki. In Ma:2020mjz,
the authors investigate the allowed parameter space in the scNMSSM, an NMSSM
extension that relaxes unification requirements at the GUT scale Das:2013ta;
Ellwanger:2014dfa; Nakamura:2015sya; Wang:2018vxp, also known as NUHM, which
contains in total 5 scalar particles; if CP is conserved, 3 are CP-even and 2
CP-odd. The authors include various bounds on the models parameter space, and
show the allowed scan points and predictions for the above channels for
various scalar combinations. We display their results in figure 14. We see
that various scenarios exist where light scalars can be realized, with masses
in the whole mass range between 10 and 60 GeV. The authors also display the
projected sensitivity limits.

Figure 14:  Allowed rates for various Higgsstrahlung processes with successive
decays into two light scalars. Top: $2\,b\,2\,\tau$ final state. Bottom:
$4\tau$ final state. Also shown are expected upper bounds for various collider
machines, with projections from Liu:2016zki. Figure taken from Ma:2020mjz.

Finally, in simple singlet extensions it is possible to test regions in the
models parameter space which can lead to a strong first-order electroweak
phase transition. Several authors have worked on this; we here show results
from Kozaczuk:2019pet, where in addition several collider sensitivity
projections are shown, including the bounds derived in Liu:2016zki. From
figure 15, it becomes obvious that $e^{+}e^{-}$ Higgs factories would be an
ideal environment to confirm or rule out such scenarios. While
Kozaczuk:2019pet rely on previous analyses of this channel, in Wang:2022dkz
the authors perform a dedicated study of this channel testing the same model
and parameter space in the CEPC setup. Their results, also displayed in figure
15, seem to confirm previous findings.

Figure 15:  Expected bounds on Higgs production via Higgs strahlung and
subsequent decay into two light scalars, in the singlet extension scenario
discussed in Kozaczuk:2019pet; Wang:2022dkz. Left: Taken from
Kozaczuk:2019pet. Right: Results presented in Wang:2022dkz, for CEPC at 250
GeV. The blue band denotes the region where a strong first-order electroweak
phase transition is possible. We see that $e^{+}e^{-}$ Higgs factories are
required on order to confirm or exclude such scenarios. LHC searches as well
as projections stem from ATLAS:2018emt; CMS:2018nsh $(b\,b\,\mu\mu)$,
CMS:2018zvv $(b\,b\,\tau\tau)$, ATLAS:2018pvw $(4\,b)$, and CMS:2019spf;
ATLAS:2015unc $(\mu\mu\tau\tau)$.

Related work, with a spontaneous breaking of the imposed $\mathbb{Z}_{2}$
symmetry, has been presented in Carena:2019une.

####  IV.1.3 Other channels

We now turn to other channels that do not focus on $Zh$ production and
successive decays involving light scalars. We briefly present two different
studies for such searches. In Chun:2019sjo, the authors investigate a slightly
different channel, i.e. tau pair-production where a light pseudoscalar is
radiated off one on the outgoing fermion lines and decays again into
$\tau^{+}\tau^{-}$, leading to a 4 $\tau$ final state. They are considering a
type X 2HDM, which in addition allows them to explain the current discrepacy
between theoretical prediction and experiment for the anomalous magnetic
momentum of the muon Muong-2:2021ojo. They perform a detailed study including
background and determine 2 and 5 $\sigma$ countours in the
$\left(m_{A},\,\tan\beta\right)$ plane, where $\tan\beta$ denotes the ratio of
the vevs of the two doublets. Their results are shown in figure 16. We see
that even for $500\,{\rm fb}^{-1}$, the proposed analysis can already test the
region of interest to expain the $g_{\mu}-2$ anomaly, including a possible
discovery in that region of parameter space for $2\,{\rm ab}^{-1}$.

Figure 16:  Exclusion and discovery regions in the 2HDM type X model, in the
$\left(m_{A},\,\tan\beta\right)$ plane. The color region additionally explains
the current $g_{\mu}-2$ discrepancy. Regions above the respective lines are
excluded. Taken from Chun:2019sjo.

It is also interesting to investigate models with give the possibility of
light charged scalars. A corresponding study has been performed in
Akeroyd:2019mvt, where the authors consider charged scalar pair-production
within a 3HDM, with successive decays into $c\,\bar{c}\,b\,\bar{b}$ final
states. The authors perform a detailed study and present their results in the
1 and 2 b-jet tagged category, as a function of light scalar mass and charm
tagging efficiency. We show the corresponding significances in figure 17, for
a com energy of 240 GeVand an integrated luminosity of $1\,{\rm ab}^{-1}$.
Depending on the charm tagging efficiency, the authors predict that relatively
large significances can be reached throughout the models parameter space.

Figure 17:  Significances as a function of charged scalar mass and charm
tagging efficiency at an 240 GeV CEPC, at an integrated luminosity of $1\,{\rm
ab}^{-1}$, within a 3HDM as presented in Akeroyd:2019mvt, considering a
$c\bar{c}b\bar{b}$ final state. Figures taken from that reference.

###  IV.2 Cross section predictions

Inspired by possible low-mass excesses in at LEP ALEPH:2006tnd and CMS
CMS:2018cyk, in Heinemeyer:2021msz several models are fitted to these excesses
that contain singlet and doublet extensions of the SM scalar sector; in
particular, they consider models with an additional doublet as well as a
(complex) singlet, labelled N2HDM and 2HDMs, respectively. For both models, as
well as varying $\tan\beta$ ranges (where $\tan\beta$ denotes the ratio of the
vevs in the 2HDM part of the models), the authors investigate the possibility
to explain the observed accesses and give rate predictions for a $250\,{\rm
GeV}$ collider with a total luminosity of $\mathcal{L}\,=\,2\,{\rm ab}^{-1}$.
We display their results in figure 18. We see that also other final states for
the $h$ decay, as e.g. $\tau^{+}\tau^{-},\,gg,$ or $W^{+}W^{-}$ can render
sizeable rates. Related work concentrating on the N2HDM can be found in
Biekotter:2022jyr.

Figure 18:  Left: Points in the 2HDMs that agree with both CMS and LEP excess
and which can be probed at the ILC. Right: predicted rates in the 2HDMS and
N2HDM at 250 GeVusing full target luminosity.

##  V Other center of mass energies

The FCC-ee and CEPC colliders are supposed to also run with a center-of-mass
energy of $\sim\,160\,{\rm GeV}$, already tested at LEP. In analogy to figure
9, in figure 19 we show again cross section predictions for the process $Z\,h$
in dependence of the mass of $h$, assuming a SM-like scalar. Note we here
assume onshell production of $Z\,h$, which leads to a hard cutoff for
$M_{h}\,\sim\,70\,{\rm GeV}$. Detailed studies should in turn assume
contributions from offshell $Z$s and $h$s as well.

Figure 19:  As figure 9, for a com of 160 GeV. We assume onshell final states.

We see that for this lower com energy, there is basically no contribution to
the $\nu_{\ell}\,\bar{\nu}_{\ell}h$ final state that does not originate from
$Z\,h$. Using FCC-ee target luminosity for this energy, and again assuming a
general suppression factor $\,\sim\,0.1$ stemming from signal strength, we
expect up to $10^{6}$ events depending on the mass of the additional scalar.

For this center-of-mass energy, several searches exist which have already been
performed at LEP and are summarized in OPAL:2002ifx; ALEPH:2006tnd,
concentrating on $Z\,h$, $h_{1}\,h_{2},$ and $h_{1}\,h_{1}\,h_{1}$ final
states, which could be further pursued in future collider studies. We want to
note that the luminosity at FCC-ee at this center of mass energy is exceeding
LEP luminosity by several orders of magnitude.

Finally, we present a study that investigates various types of 2HDMs
containing several neutral scalars Azevedo:2018llq, for a collider energy of
350 GeV. The authors perform a scan of the allowed parameter space and render
predictions for the Higgs-strahlung process as well as
$\nu_{\ell}\,\bar{\nu}_{\ell}h$ final states with the scalar decaying into
$b\,\bar{b}$ pairs. We show their results in figure 20.

Figure 20:  Achievable rates for various light scalar production modes at an
$e^{+}e^{-}$ collider with a com energy of 350 GeV, in various 2HDM variant
models. Figure taken from Azevedo:2018llq.

##  VI Conclusion

In this overview, I have presented several models and searches that
investigate the sensitivity of future $e^{+}e^{-}$ machines for scalars with
masses $\lesssim\,125\,{\rm GeV}$. This is based on several talks I have
recently given and is thereby not meant to be inclusive. I have pointed to
models that allow for such light scalars, as well as several references that
either provide rates or pursue dedicated studies. I have also pointed to the
connection of low-scalar searches at such colliders and the electroweak phase
transition within certain models. My impression is that further detailed
studies are called for, with a possible focus on so-called Higgs factories
with center-of-mass energies around 240-250 GeV.

## Acknowledgements

I thank Sven Heinemeyer and the conveners of the CEPC workshop for inspiring
me to set up an overview on these models. Several authors of the references
cited here were also helpful in answering specific questions regarding their
work.

## References

  * (1)


We present an argument from Moishe Kohan [62]. First, we show that any smooth
map $f\colon X\to Y$ is upper Lipschitz. Take any $p,q\in X$ and let
$c\colon[0,d_{X}(p,q)]\to X$ denote the unit-speed parameterization of a
shortest path from $p$ to $q$. Then

$\displaystyle d_{Y}(f(p),f(q))$
$\displaystyle{\leq}\int_{0}^{d_{X}(p,q)}\|(f\circ
c)^{\prime}(t)\|dt=\int_{0}^{d_{X}(p,q)}\|(Df)(c(t))c^{\prime}(t)\|dt$
$\displaystyle\leq\sup_{x\in
X}\|(Df)(x)\|\cdot\int_{0}^{d_{X}(p,q)}\|c^{\prime}(t)\|dt=\sup_{x\in
X}\|(Df)(x)\|\cdot d_{X}(p,q).$

Furthermore, $\beta:=\sup_{x\in X}\|(Df)(x)\|<\infty$ by compactness.

To prove lower Lipschitz, we further assume that $f\colon X\to Y$ is an
embedding. To this end, we first show that $f$ is locally lower Lipschitz in
the sense that there exist $\alpha,\epsilon>0$ such that
$d_{Y}(f(p),f(q))\geq\alpha d_{X}(p,q)$ whenever $d_{X}(p,q)<\epsilon$. To
accomplish this, we take $Z:=\operatorname{im}(f)$ and show that the inverse
map $f^{-1}\colon(Z,d_{Y})\to(X,d_{X})$ is $\delta$-locally $L$-Lipschitz, in
which case we may take $\alpha:=1/L$ and $\epsilon:=\delta/\beta$. Our
approach uses Riemannian tools to analyze arbitrary $u,v\in Z$ with
$d_{Y}(u,v)$ sufficiently small, and so we start by constructing a compact
submanifold with boundary $K\subseteq Y$ that contains $Z$ along with all
length-minimizing geodesics in $Y$ between sufficiently close points in $Z$.
Since $Z$ is a compact submanifold of $Y$, it has a normal injectivity radius
$r>0$. That is, letting $B_{r}\nu_{Z}$ denote the members of the normal bundle
$\nu_{Z}$ of $Z$ in $Y$ with norm smaller than $r$, then the restriction of
the normal exponential map $\operatorname{exp}_{Z}\colon\nu_{Z}\to Y$ to
$B_{r}\nu_{Z}$ is a diffeomorphism onto its image $N\subseteq Y$, which in
turn is an open tubular neighborhood of $Z$. Invert this diffeomorphism and
project onto $Z$ to obtain a smooth retraction ${g}\colon N\to Z$. Applying
$\operatorname{exp}_{Z}$ to the closure of $B_{r/2}\nu_{Z}$ gives a compact
submanifold with boundary $K\subseteq N$. Next, we find $\delta>0$ such that
$d_{K}(u,v)=d_{Y}(u,v)$ for every $u,v\in Z$ with $d_{K}(u,v)<\delta$. By
Theorem 6.17 in [66], every $z\in Z$ is contained in an open set
$B_{z}\subseteq K$ that is geodesically convex. This means that for every
$u,v\in B_{z}$, the distance $d_{Y}(u,v)$ equals the length of the unique
shortest path in $Y$ from $u$ to $v$, which in turn is entirely contained in
$B_{z}\subseteq K$, and so $d_{K}(u,v)=d_{Y}(u,v)$. Considering
$\\{B_{z}\\}_{z\in Z}$ is an open cover of the compact metric space
$(Z,d_{K})$, then Lebesgue’s number lemma produces $\delta>0$ such that every
$u,v\in Z$ with $d_{K}(u,v)<\delta$ necessary satisfies $u,v\in B_{z}$ for
some $z\in Z$, in which case $d_{K}(u,v)=d_{Y}(u,v)$. The previous argument
then gives that the restriction of $f^{-1}\circ g\colon N\to X$ to $Z$ is
$\delta$-locally $L$-Lipschitz for some $L<\infty$, and so $f^{-1}$ is
$\delta$-locally $L$-Lipschitz, as desired.

Finally, consider the set $C$ of $(p,q)\in X^{2}$ for which
$d_{X}(p,q)\geq\epsilon$. Then compactness gives

$R:=\sup_{(p,q)\in C}\frac{d_{X}(p,q)}{d_{Y}(f(p),f(q))}<\infty.$

Since $f$ is $\epsilon$-locally $\alpha$-lower Lipschitz, it follows that $f$
is $\min\\{\alpha,1/R\\}$-lower Lipschitz.

##  Appendix D Proof of Proposition 29

First, we note that $L|_{M}$ is not injective precisely when there exist
$x,y\in M$ with $x\neq y$ such that $L(x-y)=0$. Similarly, $L|_{M}$ is not a
smooth immersion precisely when there exists $x\in M$ and a nonzero $v\in
T_{x}M$ such that $Lv=0$. Letting $W$ denote the set of all such $x-y$ and
$v$, then by Proposition 4.22 in [65], $L|_{M}$ is not a smooth embedding
precisely when there exists $w\in W$ such that $Lw=0$. Consider the set $P$ of
$(L,w)$ for which $Lw=0$ and $w\in W$. Then $L|_{M}$ is a smooth embedding
precisely when $L$ does not reside in the projection $\pi_{L}(P)$. Lemma 2.8
in [44] gives

$\operatorname{dim}\pi_{L}(P)\leq\operatorname{dim}P\leq\operatorname{dim}W+\sup_{w\in
W}\operatorname{dim}\\{L:(L,w)\in P\\}.$

Meanwhile, $\operatorname{dim}W\leq 2k$ and
$\operatorname{dim}\\{L:Lw=0\\}=m(n-1)$ for every $w\neq 0$. Thus,
$\operatorname{dim}\pi_{L}(P)\leq 2k+m(n-1)$, which is strictly less than $mn$
when $m>2k$. It follows that a generic choice of $L$ does not reside in
$\pi_{L}(P)$.

##  Appendix E Proof of Proposition 49

For (a), choose $\epsilon>0$. Given $f\in L^{2}(G)$ and $u\in H$, let
$f\otimes u\in L^{2}(G;H)$ denote the function defined by $(f\otimes
u)(g)=f(g)u$ for $g\in G$. Such pure tensors span a dense subspace of
$L^{2}(G;H)$ by Theorem II.10 of [81]. (Here, we use the separability of
$L^{2}(G)$; see statements (16.2) and (16.12) in [55].) Find
$f_{1},\dotsc,f_{m}\in L^{2}(G)$ and unit norm $u_{1},\dotsc,u_{m}\in H$ with
$\|\varphi-\sum_{j=1}^{m}f_{j}\otimes u_{j}\|{<\frac{\epsilon}{3}}$. By
Proposition 2.41 of [48], there is a neighborhood $V$ of $1$ in $G$ such that
${\|L_{g}f_{j}-f_{j}\|}{<\frac{\epsilon}{3m}}$ whenever $g\in V$. For such
$g$, the triangle inequality implies

$\displaystyle\|L_{g}\varphi-\varphi\|$
$\displaystyle\leq\bigg{\|}L_{g}\Big{(}\varphi-\sum_{j=1}^{m}f_{j}\otimes
u_{j}\Big{)}\bigg{\|}+\bigg{\|}\sum_{j=1}^{m}(L_{g}f_{j}-f_{j})\otimes
u_{j}\bigg{\|}+\bigg{\|}\sum_{j=1}^{m}f_{j}\otimes u_{j}-\varphi\bigg{\|}$
$\displaystyle\leq 2\bigg{\|}\varphi-\sum_{j=1}^{m}f_{j}\otimes
u_{j}\bigg{\|}+\sum_{j=1}^{m}\|L_{g}f_{j}-f_{j}\|<\epsilon.$

Then for $g,h\in G$ with $h\in gV$, it holds that $\|L_{h}\varphi-
L_{g}\varphi\|=\|L_{g^{-1}h}\varphi-\varphi\|{<\epsilon}$.

Part (b) is trivial when $\varphi=0$ or $\psi=0$, and so we assume otherwise.
Continuity follows from (a), since

$|\langle L_{h}\varphi,\psi\rangle-\langle
L_{g}\varphi,\psi\rangle|\leq\|L_{h}\varphi-L_{g}\varphi\|\|\psi\|.$

It remains to prove our function vanishes at infinity.

Given a subset $E\subseteq G$, let $\mathbf{1}_{E}$ denote its indicator
function. We claim that for every $\epsilon>0$, there is a compact set
$K\subseteq G$ such that $\|\mathbf{1}_{K^{c}}\varphi\|<\epsilon$. Indeed, the
function $g\mapsto\|\varphi(g)\|$ belongs to $L^{2}(G)$, so we can approximate
it within $\epsilon$ by some $f\in C_{c}(G)$. Let $K\subseteq G$ be a compact
set outside of which $f$ vanishes. Then

$\|\mathbf{1}_{K^{c}}\varphi\|^{2}\leq\int_{K^{c}}\|\varphi(g)\|^{2}\,dg+\int_{K}\big{|}\|\varphi(g)\|-f(g)\big{|}^{2}\,dg=\int_{G}\big{|}\|\varphi(g)\|-f(g)\big{|}^{2}\,dg<\epsilon^{2},$

as desired.

Choose $\epsilon>0$. By the above, there are compact sets
$K_{1},K_{2}\subseteq G$ such that
$\|\mathbf{1}_{K_{1}^{c}}\varphi\|<\frac{\epsilon}{2\|\psi\|}$ and
$\|\mathbf{1}_{K_{2}^{c}}\psi\|<\frac{\epsilon}{2\|\varphi\|}$. Then

$K_{2}K_{1}^{-1}=\\{gh^{-1}:g\in K_{2},h\in K_{1}\\}\subseteq G$

is compact. Given $g\not\in K_{2}K_{1}^{-1}$, we show $|\langle
L_{g}\varphi,\psi\rangle|<\epsilon$. Expand

$\displaystyle\langle L_{g}\varphi,\psi\rangle$
$\displaystyle=\big{\langle}L_{g}\varphi,\mathbf{1}_{K_{2}}\psi\big{\rangle}+\big{\langle}L_{g}\varphi,\mathbf{1}_{K_{2}^{c}}\psi\rangle$
$\displaystyle=\big{\langle}\mathbf{1}_{K_{1}}\varphi,\mathbf{1}_{g^{-1}K_{2}}L_{g^{-1}}\psi\big{\rangle}+\big{\langle}\mathbf{1}_{K_{1}^{c}}\varphi,L_{g^{-1}}(\mathbf{1}_{K_{2}}\psi)\big{\rangle}+\big{\langle}L_{g}\varphi,\mathbf{1}_{K_{2}^{c}}\psi\big{\rangle}.$

Since $g\not\in K_{2}K_{1}^{-1}$, the sets $K_{1}$ and $g^{-1}K_{2}$ are
disjoint. Then the first term above vanishes, and the triangle inequality
yields

$\displaystyle|\langle L_{g}\varphi,\psi\rangle|$
$\displaystyle\leq\big{|}\big{\langle}\mathbf{1}_{K_{1}^{c}}\varphi,L_{g^{-1}}(\mathbf{1}_{K_{2}}\psi)\big{\rangle}\big{|}+\big{|}\big{\langle}L_{g}\varphi,\mathbf{1}_{K_{2}^{c}}\psi\big{\rangle}\big{|}$
$\displaystyle\leq\|\mathbf{1}_{K_{1}^{c}}\varphi\|\|L_{g^{-1}}(\mathbf{1}_{K_{2}}\psi)\|+\|L_{g}\varphi\|\|\mathbf{1}_{K_{2}^{c}}\psi\|$
$\displaystyle=\|\mathbf{1}_{K_{1}^{c}}\varphi\|\|\mathbf{1}_{K_{2}}\psi\|+\|\varphi\|\|\mathbf{1}_{K_{2}^{c}}\psi\|\leq\|\mathbf{1}_{K_{1}^{c}}\varphi\|\|\psi\|+\|\varphi\|\|\mathbf{1}_{K_{2}^{c}}\psi\|<\epsilon,$

as desired.