# Masked Autoencoders Are Articulatory Learners

###### Abstract

Articulatory recordings track the positions and motion of different
articulators along the vocal tract and are widely used to study speech
production and to develop speech technologies such as articulatory based
speech synthesizers and speech inversion systems. The University of Wisconsin
X-Ray Microbeam (XRMB) dataset is one of various datasets that provide
articulatory recordings synced with audio recordings. The XRMB articulatory
recordings employ pellets placed on a number of articulators which can be
tracked by the microbeam. However, a significant portion of the articulatory
recordings are mistracked, and have been so far unusable. In this work, we
present a deep learning based approach using Masked Autoencoders to accurately
reconstruct the mistracked articulatory recordings for 41 out of 47 speakers
of the XRMB dataset. Our model is able to reconstruct articulatory
trajectories that closely match ground truth, even when three out of eight
articulators are mistracked, and retrieve 3.28 out of 3.4 hours of previously
unusable recordings.

Index Terms—  X-ray microbeam, articulatory, data reconstruction, masked
autoencoder, deep learning

##  1 Introduction

Articulatory data have been foundational in helping speech scientists study
the articulatory habits of speakers that can differ between individuals, and
across languages and dialects of the same language. At present, specialized
equipment such as X-ray Microbeam (XRMB) [1] Electromagnetic Articulometry
(EMA) [2] and real-time Magnetic Resonance Imaging (rt-MRI) [3] is needed to
observe articulatory movements directly to help researchers and clinicians
understand these habits. Regardless of the technique, current technologies
used in articulatory recordings sometime fail to correctly capture one or more
of the articulators, resulting in mistracked segments. Given the considerable
time and cost involved in data collection and the potential prolonged exposure
to harmful signals, these mistracked pellets are seldom re-recorded leaving
some recordings unusable.

There has been considerable work to develop techniques to reconstruct missing
articulatory recordings. In [4], the authors proposed an algorithm for
recovering missing data by learning a density model of the vocal tract shapes.
Their model was limited to recovering only one mistracked articulator at a
time. Also, their approach requires handcrafting the model parameters for
every speaker and every articulator. In [5, 6], a Kalman smoother and maximum
a-posteriori estimator fills in the missing samples. Their work was also
limited to one articulator at a time, and was additionally dependent on the
length of the mistracked samples.

Our approach does not suffer from the same limitations of previous works. The
model hyper-parameters do not require any handcrafting for different speakers,
and one model can predict any missing articulator. As a result, our model is
easily applicable for the majority of the dataset. We are also not limited to
reconstructing one missing articulator, or limited to a certain duration of
mistracked segments, but are able to reconstruct up to three missing
articulators that follow the ground truth closely for the entire duration of
the recording.

We approach the problem as a masked reconstruction problem. Recently, there
has been some major breakthroughs in image reconstruction using masked
autoencoders [7], where an image can be realistically reconstructed with 90%
of it being masked in patches. This work follows a similar approach with
articulatory data, where a portion of an input frame is masked and then fed to
an autoencoder, which then reconstructs the entire frame including the masked
portions. We explain our approach in greater detail in section 3. Section 2
outlines the dataset description. We demonstrate test set results in section
4, and discuss the limitations of the proposed approach in 5. We end with a
conclusion and discussion about future directions in section 6.

##  2 Dataset description

In this study, we use the University of Wisconsin XRMB dataset. However, there
is no reason the proposed approach cannot be applied to other datasets.

###  2.1 XRMB dataset

The XRMB dataset contains articulatory recordings synced with audio
recordings. Each speaker had 8 gold pellets glued on 8 articulators, upper lip
(UL), lower lip (LL), tongue tip (T1), tongue blade (T2), tongue dorsum (T3),
tongue root (T4), mandible incisor (MNI), and (parasagittally placed) mandible
molar (MNM). The X-Y coordinates for every pellet were tracked through time
and sampled at different sampling rates. All pellet trajectories (PTs) were
then resampled to have a sampling rate of 160 samples per second.

We only use data from 41 speakers from the XRMB dataset. These speakers have
enough uncorrupted data for the model to train and generalize. The six
speakers not used (JW29, JW32, JW33, JW42, JW44, JW60) do not have enough
uncorrupted data for training.

All recordings start with a tone to show the speaker when to start speaking,
and some recordings contain trailing silences at the end of the recording, and
mid-sentence silences between different utterances. We trim the silences from
these utterances. Some tasks do not include speech (swallowing, …etc). We do
not include nonverbal tasks in the dataset.

Table 1 shows the amount of mistracking in different subsets of the XRMB
dataset. The third column shows the duration of data if we limit ourselves to
only using recordings without any mistracking. Such a limitation is reasonable
to ensure that the recordings can be broken down into contiguous frames. Most
discarded recordings contain only a few milliseconds of mistracking, but are
rendered useless. Figure 1 shows that more than 25% of the files with
mistracking present have multiple PTs concurrently mistracked.

Table 1: Percentage and duration of clean and mistracked data in different
subsets of the XRMB dataset.

Dataset Subset | Total Duration Of Recordings |  | Total Duration Of Recordings
---
Without Mistracking
| % Recordings With
---
Mistracking
47 Speakers | 14.36 hrs | 8.62 hrs |  37.4%
| 41 Speakers used
---
in this study
12.68 hrs | 8.46 hrs | 30.3%
| 41 Speakers used in this study
---
using only the verbal tasks
with extended silences removed
10.6 hrs | 7.2 hrs | 30.3%

$0$$20$$40$$60$$80$$100$PercentageOneTwoThreeMore Than Three

Fig. 1: Percentage of total duration of recordings with different degrees of
mistracking to the total duration of recordings with mistracked samples in the
XRMB dataset.

###  2.2 Training and test sets

The training data was created through a sliding window on all recordings.
After the frames were created, only the frames without any mistracking were
kept and the rest were disregarded. The test set was created similarly.
Instead of randomly sampling the test set from the entire dataset, a set task
list was handcrafted, were all the recordings corresponding to these tasks are
used in the test set. This was done to ensure that the test set covers the
span of all verbal tasks in the dataset, is a good representation of the out
of sample performance, and is as consistent as possible across speakers. Since
not all speakers performed all tasks, when a speaker didn’t perform a task in
the predetermined task list, a similar task was chosen instead. Separate
training and test datasets were created for every speaker, and a different
model was trained for each speaker.

##  3 Approach

As mentioned in Section 1, we follow a similar approach to the work put forth
in [7], where masked autoencoders reconstructed images with large amounts of
masking. Inspired by this work, we designed our deep learning model using
time-dilated dense layers and Bi-directional Gated Recurrent Network layers
(Bi-GRUs).

We use a 200 sample input frame. The input frames are masked randomly. For
each batch, a masking layer samples an integer between 1 and 8 uniformly $N$
times, with $N$ being a hyper-parameter. Unique samples represent the index of
PTs to be masked and replaced by a trainable mask token. This means that the
hyper-parameter $N$ controls the maximum amount of masking during training.

Unlike [7], we don’t remove the masked patches from the input to the encoder
and provide positional information to the input of the decoder. Instead we
mask in place. This ensures that separate channels in the model keeps the
information about the identity of different PTs, masked or unmasked.

The model was implemented with TensorFlow-Keras machine learning framework and
trained on an NVIDIA Titan Xp GPU. We used Adam optimizer and mean absolute
error loss. We employed early stopping with a patience of 50 epochs(the
training is terminated when the test set loss isn’t improved for 50 epochs).
Even though the aim of the model is to reconstruct masked PTs, the loss is
calculated on the entire output tensor to ensure that the model keeps the
information about inter-dependencies between different PTs throughout. We
trained 8 models for every speaker, varying $N$ from 1 to 8. In the end, we
choose only one model for every speaker for inference based on test set
performance.

##  4 Results

The results in this section approximate worst-case conditions, where one or
more PTs is mistracked for the entire test set. The Pearson Product Moment
Correlation (PPMC) score between the predicted and ground truth PTs is
calculated and presented. In practical settings, PTs are commonly only
mistracked for a short duration (50-500 ms)[1]. However, recordings where one
PT is entirely mistracked for the full duration of the recordings exist, and
hence we use this as our test case. Note that fixing short segments of
mistracking will make entire recordings, each a few seconds long usable again.

###  4.1 Tuning hyper-parameter $N$

Empirical experiments showed that the higher the value of N, the better and
more uniformly the model performs across different degrees of masking during
testing. This means that by increasing N the model performs better with higher
masking levels during testing, even beyond the training upper-limit. However,
values of N beyond 4 usually lead to performance degradation for lower levels
of masking during testing. Due to page limitations, we will not show
comparisons between all 8 possible values of $N$, and we’ll just demonstrate
the difference between three different values for a single speaker (JW11) in
figures 2, 3 and 4.

1234567$0$$0.2$$0.4$$0.6$$0.8$$1$$0.879$$0.843$$0.776$$0.622$$0.369$$0.225$$0.109$$0.917$$0.868$$0.79$$0.636$$0.408$$0.162$$3\cdot
10^{-2}$Number of Masked PTs During TestingPPMCX-CoordinatesY-Coordinates

Fig. 2: Average PPMC scores for different levels of masking during testing for
a model trained with N = 1.

1234567$0$$0.2$$0.4$$0.6$$0.8$$1$$0.883$$0.87$$0.85$$0.808$$0.726$$0.588$$0.376$$0.928$$0.915$$0.894$$0.858$$0.791$$0.672$$0.463$Number
of Masked PTs During TestingPPMCX-CoordinatesY-Coordinates

Fig. 3: Average PPMC scores for different levels of masking during testing for
a model trained with N = 3.

1234567$0$$0.2$$0.4$$0.6$$0.8$$1$$0.847$$0.836$$0.819$$0.786$$0.724$$0.617$$0.416$$0.904$$0.894$$0.876$$0.847$$0.802$$0.717$$0.53$Number
of Masked PTs During TestingPPMCX-CoordinatesY-Coordinates

Fig. 4: Average PPMC scores for different levels of masking during testing for
a model trained with N = 5.

It should be noted that the overwhelming majority of corrupted recordings have
at most 3 concurrently mistracked PTs, and hence the best values for N will be
chosen empirically for each speaker depending on the performance of each model
when masking 1, 2 or 3 PTs concurrently.

###  4.2 Overlapping frames for the training set

We experimented with modifying the training data by allowing for 50% overlap
between adjacent frames. Overlap between frames increases the total number of
frames, but in this case the number of frames in the training data increased
beyond the theoritical limit. This is due to the fact that we are utilizing
recordings with mistracked samples by framing them and disregarding any frames
with corruption, and keeping the rest. Overlapping increases the chance for
previously unused samples to be caught in a sliding frame. Figure 5 shows test
set results for a model trained with overlapping frames and with N=3.
Comparing with Figure 3 which shows results for an identical model trained on
the same training set without overlap, we can see that allowing for overlap
between adjacent frames in the training set increased test set performance by
approximately 10 points on average.

1234567$0$$0.2$$0.4$$0.6$$0.8$$1$$0.96$$0.954$$0.946$$0.931$$0.905$$0.71$$0.172$$0.975$$0.969$$0.958$$0.936$$0.897$$0.716$$0.211$Number
of Masked PTs During TestingPPMCX-CoordinatesY-Coordinates

Fig. 5: Average PPMC scores for different levels of masking during testing for
a model trained with N = 3 and overlapping frames.

###  4.3 Sample outputs

Figure 6 shows a sample reconstruction for artificially masked data. Three PTs
where randomly chosen and artificially masked for the entire duration of the
recording (10 seconds). There is no instance in the XRMB dataset where three
PTs are concurrently mistracked for such a long duration, so this test case
provides less context to the model than actual use cases and is thus harder.
The model was still able to closely follow the ground truth values of the PTs.

Figure 7 shows a sample reconstruction for actual corrupted data. The
reconstructed trajectory of the mistracked segment matches the phonetic
transcription, for instance, the peak in T1-y after 5.4 seconds corresponds to
the $n$ sound, and the peak in T1-x at 6.0 seconds corresponds to the $\eth$
sound.

Fig. 6: Sample reconstruction of 3 artificially and concurrently masked PTs
from task 003 (reading a paragraph) by speaker JW52. Fig. 7: Sample
reconstruction of actual corrupted PTs from the utterance ”…true son of the
out of…” from task 78 by speaker JW11.

##  5 Performance Limitations

So far, we’ve shown the average PPMC score per level of masking during
testing. This view point, while useful for comparing models and hyper-
parameter tuning, does not demonstrate enough the consistency of the models’
performance in retrieving different PTs. For this, we show the results from
the same model from Figure 5 in a more comprehensive way. Figure 8 shows the
average PPMC score per PT when masking 3 PTs during training and with N=3.
Each bar shows the average PPMC of each PT over all the combinations of 3
masked PTs that specific PT appears in. The error bars show the highest and
lowest PPMC score for each PT. The average PPMC score over all the cases in
Figure 8 corresponds to the third bar in Figure 5.

ULLLT1T2T3T4MNIMNM$0$$0.2$$0.4$$0.6$$0.8$$1$$0.98$$0.85$$0.97$$0.99$$1$$1$$0.81$$0.99$$0.97$$0.92$$0.96$$0.97$$0.97$$0.95$$0.98$$0.95$Masked
PTsPPMCX-CoordinatesY-Coordinates

Fig. 8: Average PPMC scores per each PT when masking 3 PTs at a time and for a
model trained wih N=3.

It is clear from Figure 8 that the model does not handle some cases as well as
others. In some cases the model is not able to retrieve missing PTs when
closely related PTs are masked as well. For example, when UL and LL are masked
together, the model often suffers to retrieve one or both of them. Similarly
when MNI and MNM are both masked. This is also the case when masking three out
of the 4 tongue PTs (T1, T2, T3 and T4).

Fortunately, these cases are not common in the subset of 41 speakers used in
this study. Eliminating recordings with related PTs corrupted together reduces
the duration of the reconstructed dataset by 0.12 hours, meaning that we can
still increase the size of usuable data in this subset from 7.2 hours to 10.48
hours. Disregarding these cases ensures that the model can reconstruct up to 3
concurrently corrupted PTs with the worst case PPMC score above 0.7, and
average PPMC score above 0.9. Figure 9 shows that limiting the test cases
results in a more consistent performance.

ULLLT1T2T3T4MNIMNM$0$$0.2$$0.4$$0.6$$0.8$$1$$0.98$$0.87$$0.97$$0.99$$1$$1$$0.85$$1$$0.97$$0.93$$0.97$$0.98$$0.98$$0.96$$0.99$$0.97$Masked
PTsPPMCX-CoordinatesY-Coordinates

Fig. 9: Average PPMC scores per each PT when masking 3 PTs at a time and for a
model trained wih N=3. The test cases here do not include masking UL and LL,
or MNI and MNM together, or three of the 4 tongue PTs.

##  6 Conclusion And Future Work

We have shown that a Masked Autoencoder model can closely reconstruct
mistracked PTs in the XRMB dataset. We have expanded the usable portion from
7.2 hours to 10.48 hours, a 45% increase in duration.

One limitation of our approach was its inability to confidently reconstruct
concurrently mistracked related PTs. Future work should explore different
architectures to further exploit interdependencies between different
articulators. Furthermore, our data driven and speaker dependent approach
meant we were unable to reconstruct mistracking in scarce datasets. A speaker
independent approach should be invistigated in the future, along with data
augmentation techniques and utilizing corrupted recordings in the training
data.

## References

  * [1] John R Westbury,  “Speech Production Database User ’ S Handbook,”  IEEE Personal Communications - IEEE Pers. Commun., vol. 0, no. June, 1994.
  * [2] Paul W. Schönle, Klaus Gräbe, Peter Wenig, Jörg Höhne, Jörg Schrader, and Bastian Conrad,  “Electromagnetic articulography: Use of alternating magnetic fields for tracking movements of multiple points inside and outside the vocal tract,”  Brain and Language, vol. 31, no. 1, pp. 26–35, may 1987.
  * [3] Shrikanth Narayanan, Krishna Nayak, Sungbok Lee, Abhinav Sethy, and Dani Byrd,  “An approach to real-time magnetic resonance imaging for speech production,”  The Journal of the Acoustical Society of America, vol. 115, no. 4, pp. 1771–1776, mar 2004.
  * [4] Chao Qin and Miguel Carreira-Perpiñán,  “Estimating missing data sequences in x-ray microbeam recordings,”  01 2010, pp. 1592–1595.
  * [5] Sujith P and Prasanta Kumar Ghosh,  “Missing samples estimation in electromagnetic articulography data using equality constrained kalman smoother,”  in Proc. Interspeech 2014, 2014, pp. 716–720.
  * [6] P Sujith and Prasanta Kumar Ghosh,  “Maximum a-posteriori estimation of missing samples with continuity constraint in electromagnetic articulography data,”  in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014, pp. 940–944.
  * [7] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick,  “Masked autoencoders are scalable vision learners,”  in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 15979–15988.


# Manipulating multiple optical parametric processes in photonic topological
insulators

Zhen Jiang1,2,# Bo Ji1,2,# Yanghe Chen1,2 Chun Jiang1<EMAIL_ADDRESS>Guangqiang He1,2<EMAIL_ADDRESS>1State Key Laboratory of Advanced Optical
Communication Systems and Networks, Department of Electronic Engineering,
Shanghai Jiao Tong University, Shanghai 200240, China
2SJTU Pinghu Institute of Intelligent Optoelectronics, Department of
Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
#These authors contributed equally to this work.

Topological quantum optics, optical parametric process, entangled biphoton
state

††preprint: APS/1

Topological quantum optics, an emerging area of study, holds the potential to
bring about substantial enhancements for integrated quantum devices. Here we
propose integrated topological quantum devices performing various functions
including optical parametric amplification, frequency division, and frequency
entangled biphoton generation. We show two distinct edge modes corresponding
to different frequency ranges in both sandwich kagome and honeycomb
topological designs that emulate the quantum valley Hall effect. These two
topological edge modes enable two types of optical parametric processes
through four-wave mixing, specifically inter-band and intra-band cases. The
devices emulating photonic valley-Hall insulators allow the frequency division
of two transverse modes, and furthermore, enable the separation of two quantum
functionalities - optical parametric amplification and frequency entangled
biphoton state generation. More importantly, the parametric processes are
inborn topological protected, showing robustness against sharp bends and
disorders. Our proposal significantly widens the possibilities for robust,
multifunctional topological quantum devices on-chip, which may find
applications in quantum information processing.

##  I Introduction

The burgeoning field of on-chip quantum light sources has been undergoing
revolutionized development thanks to the advancements in nano-fabrication
technologies. Significant advancements in reducing size and improving
stability through photonic integrated circuits have played a pivotal role in
enabling the on-chip generation, controlling, and generation of quantum light
sources [1]. These improvements have consistently supported more complex and
expansive quantum operations, which are crucial for the progress in quantum
computing [2, 3], communication [4, 5, 6], and sensing [7, 8]. Two key aspects
of on-chip quantum light sources are the amplification of light signals and
the generation of entangled photon pairs. Implementing on-chip multifunctional
quantum capabilities simultaneously requires precise dispersion engineering
and specific materials, which, from a certain perspective, is still
challenging.

In parallel, integrating topological phases into quantum systems is enhanced
by the robust guidance and manipulation of light, and holds great potential as
a cutting-edge and promising area of research [9, 10, 11]. This approach is
key to maintaining stable generation and transporting of quantum states.
Topological phases possess a topological nature that grants quantum states
with robustness against structural imperfections and disorders. Notably, there
have been significant advancements in this field, such as the emergence of
topological quantum emitters [12, 13], topological quantum interference [14,
15], topological biphoton states [16, 17, 18], and even topological quantum
frequency combs [19, 20]. At the same time, emergent advances in topological
nonlinear optics also promise topological protection of complex nonlinear
processes [21, 22, 23, 24, 25, 26]. A significant amount of study has been
focused on the development of quantum light sources within topological optical
systems. Nevertheless, the investigation of multifunctional quantum devices in
topological photonic systems remains unexplored.

Here we demonstrate integrated topological quantum devices that perform
various functions, including optical parametric amplification (OPA), frequency
division, and the generation of entangled biphotons. We demonstrate the
existence of two separate edge modes in sandwich kagome and honeycomb
topological designs that emulate the quantum valley Hall (QVH) effect. By
employing a diamond structure, it is possible to couple two separate edge
modes to opposite branches, which allows for the separation of spatial modes.
The presence of two edge modes in kagome topological photonic crystals (TPCs)
enables four-wave mixing (FWM) processes, leading to two types of optical
parametric processes (OPPs) - one related to inter-band scenarios and the
other to intra-band scenarios. Importantly, the distinct transmission paths of
these edge modes enable the individual facilitation of quantum OPA and the
generation of continuous frequency entangled biphoton along separate branches.
Besides, we also expand our multifunctional topological quantum devices in
honeycomb valley photonic crystals (VPCs). Additionally, our quantum processes
exhibit topological protection features, showing robustness against defects
and sharp bends. Our approach expands the potential for on-chip, robust, and
multifunctional topological quantum devices, opening up new avenues for
exploration in quantum optics.

##  II Results

###  II.1 Topological edge modes in kagome lattice

The discovery of the photonic kagome lattice offers a feasible framework for
the controllable design of higher-order valley-Hall edge modes [27, 28]. Here
we explore a two-dimensional topological kagome lattice supporting the
generation and flexible control of photonic topological quantum states. As
depicted in Fig.1(a), the topological design consists of two kagome lattices
with $\rm C_{3}$ symmetry. The effective topological transition is performed
by expanding or shrinking an unperturbed kagome lattice with a lattice
constant $a=480$ nm. Figure 1(b) shows the band structures for an unperturbed
kagome lattice (grey dots) and an expanded one (blue dots). Due to the high
symmetry of kagome lattices, there appears a Dirac-like degeneracy at the two
high symmetry points ($\rm K$ and $\rm K^{\prime}$ valleys) of the Brillouin
zone [28]. The deformation of the unperturbed kagome lattice leads to a
complete photonic bandgap and band inversion mechanism. The breathing kagome
lattice exhibits three mirror symmetries: $M_{x}$ for the x axis, and
$M_{\pm}$ for the two lines obtained by rotating the x-axis by $\pm 2\pi/3$
[29]. The polarization along the $x_{i}$ axis represents the expectation value
of the position with
$p_{i}=\frac{1}{S}\int_{\mathrm{BZ}}A_{i}d^{2}\mathbf{k}$, where
$A_{i}=-i\langle\psi|\partial_{k_{i}}|\psi\rangle$ dentes the Berry connection
with $x_{i}=x,y$ [29]. The topological bulk polarization describes the shift
in the average position of the Wannier center from the center of the unit
cell. It is noted that the topological bulk difference of shrunken and
expanded kagome lattice corresponds to $P=(0,0)$ and $P=(1/3,1/3)$
respectively, which denotes a trivial and nontrivial case, respectively
(Supplementary Section I).

Figure 1: (a) A scheme of topological design is composed of two types of
silicon-based kagome lattices with $\rm C_{3}$ symmetry. (b) Band diagrams for
an unperturbed kagome lattice (grey dots) and an expanded one (blue dots).
Calculated dispersion curves for (c) sandwich TPC, (d) TPC1, and (e) TPC2,
respectively, which are composed of shrunken ($d=0.18a$) and expanded
($d=0.40a$) kagome lattices. The bottom insets show the electric field
distributions for two edge modes.

Due to the bulk–boundary correspondence, the nontrivial polarization
difference leads to topological edge states localized at boundaries between
the shrunken and expanded kagome lattices. Additionally, for a finite
structure, the distorted kagome lattice is expected to show higher-order
topological states such as zero-dimensional corner states [28]. We first
calculate the band structure for a TPC supercell composed of trivial and
nontrivial kagome lattices as illustrated in Fig.1(c). The dispersion curve
reveals the presence of two edge modes that are localized in the topological
bandgap. Furthermore, it leads to an exceptionally wide bandwidth that
surpasses 40 THz. The bottom inset illustrates that these two modes are
exclusively confined to the distinct inner interfaces between two types of
kagome lattices. This behavior arises from the presence of two boundaries in
the sandwich TPC, which correspond to two topological transitions in the case
of trivial-nontrivial-trivial topology.

We also perform dispersion calculations for two different TPCs including
inverted kagome lattices, which exhibit a single topological interface. As
depicted in Fig.1(d)-(e), both of them exhibit two edge bands located within
the topological bandgap. However, the electric field of two edge modes in each
TPC is localized at the outer and inner interfaces, respectively. Note that
the emergence of outer mode is attributed to the application of periodic
boundary conditions on the outer boundaries in the simulation model [28].
Correspondingly, exchanging two kagome lattices will lead to the reversion of
topological edge states due to the inversion of nontrivial polarization
difference [29]. The field distribution and band topologies suggest that the
two separate edge modes found in the sandwich TPC are associated with the
inner boundary states in the other two TPCs. The mode-matching behavior
simplifies the coupling edge modes from the sandwich TPC to other TPCs,
rendering the coupling process more straightforward. Due to the distinct
frequency ranges of the two edge modes, the coupling between different modes
achieves a frequency-dependent filtering capability. In other words, the
frequency division characteristic enables the realization of multifunctional
on-chip topological photonic devices, potentially finding applications in
areas such as optical transmission and light source generation.

Accordingly, we analyze a sandwich waveguide (depicted as the grey region)
alongside a diamond-shaped hybrid structure consisting of trivial (represented
by the green region) and nontrivial (represented by the orange region) kagome
lattices (Fig.2(a)). It’s important to highlight that in this design, the
edges of the rhombus exhibit distinct topological edge states as a result of
the mirror symmetry of the lattices. There are two allowed edge modes
simultaneously with different frequency ranges for the sandwich TPC region.
However, for the diamond-shaped hybrid structure, the specific structure
contains two edge modes characterized by distinct frequency ranges. These edge
modes can be efficiently transmitted to the left and right branches with the
frequency range of $f>193$ THz and $f<193$ THz, respectively.

To get deeper insights into the characteristics of topological edge modes, we
simulate the field profiles in this diamond-shaped structure at different
frequencies. As illustrated in Fig.2(c)-(f), the energy couples to the
opposite branch corresponding to different pump frequencies. Two probes are
positioned at the output ports of two branches to monitor field intensity.
Figure 2(b) shows the simulated transmission spectra of the light, which
clearly reveals the frequency division functionality of our design. Note that
the presence of a frequency gap, where no energy power is detected at either
of the output ports, arises from the competition between the two edge modes.
Correspondingly, as depicted in Fig.2(d), the light is unable to interact with
any branch at this specific frequency of around 193 THz. It is worth
mentioning that this dichroic mirror behavior has the potential to facilitate
a range of innovative topological functionalities.

###  II.2 Multiple OPPs in topological device

Thanks to the fascinating functionalities of our diamond-shaped topological
structure, we expect the stable generation and flexible manipulation of
topological quantum states. As a result of the third-order nonlinearity of
silicon, the nonlinear FWM processes generated in sandwich TPCs may lead to
the signal and idler photons. The energy and momentum conversion equations
that govern the FWM processes are defined as
$2\omega_{p}=\omega_{s}+\omega_{i}$ and $2\bm{k_{p}}=\bm{k_{s}}+\bm{k_{i}}$,
where $\omega_{p,s,i}$ and $\bm{k}_{p,s,i}$ represent the frequencies and
wavevectors of the pump, signal, and idler, respectively. Generally, the
Hamiltonian for the FWM process in our topological waveguide can be written as

$\hat{H}_{NL}=\hat{H}_{\mathrm{SPM}}+\hat{H}_{\mathrm{XPM}}+\hat{H}_{\mathrm{FWM}},$
(1)

where $\hat{H}_{\mathrm{SPM}}$, $\hat{H}_{\mathrm{XPM}}$ and
$\hat{H}_{\mathrm{FWM}}$ denote the self-phase modulation (SPM), cross-phase
modulation (XPM) and FWM processes, respectively. The SPM and XPM terms
influence the oscillation process Due to the frequency division of our
diamond-shaped topological structure, the left and right branch corresponds to
the OPA process and entangled biphoton generation respectively (see
Supplementary Section II for details).

By matching the frequencies utilized in FWM processes with the operating
bandwidths of topological edge states, it becomes possible to implement
topological protection for entangled biphoton states [16, 17, 18], and even
quantum frequency combs [19, 20]. The dispersion engineering of topological
edge states offers a possible method for manipulating FWM processes within the
topological bandgap [16]. Note that the energy conversion of FWM processes is
related to phase-matching intensity $\rm PM=\operatorname{sinc}({\frac{\Delta
kL}{2}})$ [30].

Figure 2: (a) A scheme of a topological device composed of a sandwich
waveguide (grey region) and a diamond-shaped hybrid structure. (b) Normalized
field of the light monitored by two probes positioned at the output ports of
two branches of the diamond-shaped structure. (c)-(f) Field profiles for edge
modes at different frequencies in our topological device.

The phase-matching intensity distribution of FWM processes is depicted in
Fig.3(a), demonstrating three distinct phase-matching scenarios. The two main
bright regions correspond to the intra-band OPPs of the two edge modes
themselves. However, besides the intra-band OPP, an additional phase-matching
condition is also observed, corresponding to the inter-band OPP between two
edge modes. The nonlinear interactions between two edge modes result in mode
conversion, which can lead to significant correlations between different
transverse modes.

By pumping the sandwich topological waveguide with the frequency of 188 THz,
we can calculate the joint spectral amplitude (JSA) of the biphoton state
generated from the FWM process. Such a biphoton state can be given by

$|\Psi\rangle=\int\int
d\omega_{s}d\omega_{i}{\mathcal{A}}\left(\omega_{s},\omega_{i}\right)\hat{a}_{s}^{\dagger}{}{(\omega_{s})}\hat{a}_{i}^{\dagger}{}{(\omega_{i})}|0\rangle,$
(2)

where $\hat{a}^{\dagger}{}_{\omega_{s}}$ and
$\hat{a}^{\dagger}{}_{\omega_{s}}$ are creation operators for photons, and
${\mathcal{A}}\left(\omega_{s},\omega_{i}\right)$ is the JSA. The JSA is
governed by
${\mathcal{A}}\left(\omega_{s},\omega_{i}\right)=\alpha(\frac{\omega_{s}+\omega_{i}}{2})\operatorname{sinc}({\frac{\Delta
kL}{2}})$, where the pump spectrum $\alpha(\frac{\omega_{s}+\omega_{i}}{2})$
and joint phase-matching spectrum $\operatorname{sinc}({\frac{\Delta
kL}{2}})$. Our pump is Gaussian with a frequency center of $f_{p}=188$ THz and
full width at half-maximum of $\Delta f_{p}=115$ GHz.

Consequently, the JSA characterizing biphoton state generated in sandwich TPCs
is plotted in Fig.3(b), where the main intensity region along the diagonal
axis denotes a strong signal-idler correlation in the frequency domain [16].
Notably, two extra bright spots are positioned symmetrically above and below
the central line, suggesting the presence of inter-band OPP. The spot-like
phase-matching intensity distribution is also proven to be a perfect case for
a heralded single photon generator [30], and also, its purity can be improved
by machine-learning methods [31, 32]. These points indicate the existence of
frequency correlations resulting from the additional inter-band OPP
interaction between edge mode 1 and edge mode 2. Note that the signal mode
frequency is larger than 193 THz, therefore the generated signal can pass
through the left branch while the generated idler passes through the right
one. The potential phase-matching conditions between two different topological
edge states promise many effective solutions for manipulating photonic
topological quantum states.

As a conjugate variable of frequency, we can obtain the joint temporal
amplitude (JTA) of biphotons from the Fourier transformation of JSA by
$\tilde{\mathcal{A}}(t_{s},t_{i})={\mathcal{F}}[\mathcal{A}(\omega_{i},\omega_{s})]$
[32]. The peak intensity of the JTA is located at
$\Delta\tau_{y}=\Delta\tau_{z}=0$, indicating a relative phase value of
$\phi=0$ (Fig. 3(c)). The biphoton state exhibits a signal-idler time
correlation with a bandwidth of 10 ps. Furthermore, we analyze the photonic
topological quantum states by implementing quantum state tomography at a set
of bases. For the construction of quantum state tomography, the real part of
the density matrix is given by
$Re(\rho)=\left|\Psi\right\rangle\left<\Psi\right|$, utilizing a basis set
comprising 49 frequency divisions. We calculate three parts of the density
matrix $Re(\rho)$ from the JSA of the quantum state in Fig.3(d)-(f)
(corresponding to three dashed boxes in Fig.3(b)).

Figure 3: (a) Phase-matching intensity distribution of FWM processes for the
sandwich TPC. (b) JSA distribution and (c) corresponding JTA distribution
characterizing the biphoton state generated in the sandwich TPC. (d)-(f) Three
parts of the density matrix $Re(\rho)$ from the JSA of the quantum state.

###  II.3 Inter-band OPP: tunable OPA

Our topological scheme supporting multiple OPPs offers a new approach to
manipulating quantum functional devices. Here we implement an OPA through FWM
with inter-band OPP, the frequency division of the diamond-shaped structure
leads to spatial separation of signal photons. This spatial separation
behavior allows the direct extraction of amplified optical signals since the
generated signal could pass through the left branch of the diamond-shaped
structure ($f_{s}>193$ THz). We initially investigate the frequency
distribution of the signal and idler modes resulting from the inter-band OPP
with different pump frequencies. As shown in Fig.4(a), the tunable range of
the signal light extends from 193.5 THz to 196 THz, achieving a tunable
characteristic of 2.5 THz.

Typically, signal and strong pump modes are coupled into the topological
waveguide, in which the signal power is amplified via degenerate FWM [33]. The
FWM gain coefficient is given by $g=\sqrt{\gamma P_{p}\Delta k-\left(\Delta
k/2\right)^{2}}$ with effective nonlinearity
$\omega_{p}{n}_{2}/cA_{\mathrm{eff}}$, in which $n_{2}$ is Kerr nonlinearity,
$A_{\mathrm{eff}}$ is the nonlinear effective area. The effective
amplification in the waveguide requires strict adherence to a specific phase-
matching condition due to the coherent nature of parametric interaction.
Figure 4(b) illustrates the FWM gain coefficient corresponding to intra-band
OPP with varying pump frequencies for a $400a$ length topological waveguide (1
W pump power). The intra-band OPP allows a super-narrow bandwidth of large
amplification with a half maximum full width (FWHM) of around 8 GHz. Also, the
device can further achieve tuning of the amplification region by varying the
input pump light, ranging from 193.5 THz to 196 THz. This tuning process is
nonlinear due to the phase-matching condition. At the central frequency of the
amplification region, the FWM gain coefficient of up to 30 dB/cm can be
achieved, and the FWM gain is flat during the tuning of the pump frequency.
Such OPA with tunable narrow bandwidth can be applied for amplifying signals
from a single photon source.

Figure 4: (a) Frequency distribution of the signal and idler modes resulting
from the inter-band OPP with different pump frequencies. (b) FWM gain
coefficient corresponding to intra-band OPP with varying pump frequencies for
a $400a$ length topological waveguide (1 W pump power) (c) Signal gain as
functions of pump power for the waveguide length with $L=200a$, $400a$, and
$600a$, respectively.

Consider the pump wave undergoes SPM, simultaneously causing cross-phase
modulation XPM on both the signal and idler modes. Therefore, the nonlinear
phase mismatch caused by SPM and XPM should be taken into account, and the
updated phase mismatch is given by $\Delta k_{all}=2\gamma
P_{\mathrm{p}}-\Delta k$ [34], where $\gamma=\omega_{p}n_{2}A_{eff}$ is the
effective nonlinearity of the topological waveguide, $A_{eff}$ is the
effective nonlinear interaction area. Neglecting the optical propagation loss,
the observed signal gain generated via FWM for inter-band OPP can be written
as [35]

$G_{\mathrm{s}}=\frac{P_{\mathrm{s}}^{\mathrm{out}}}{P_{\mathrm{s}}^{\mathrm{in}}}=1+\left(\frac{\gamma
P_{\mathrm{p}}}{g}\sinh(gL)\right)^{2},$ (3)

In Fig.4(c), we plot the signal gain as functions of pump power for the
waveguide length with $L=200a$, $400a$, and $600a$, respectively. For a
topological waveguide with $L=400a$, a peak gain of 5dB can be achieved when
the pump power exceeds 4W. Note that when the gain of an OPA is large,
generated signal photons can undergo significant amplification, reaching
macroscopic levels through a phenomenon known as optical parametric
generation. The expected number of photons at the output is given by $\langle
n\rangle=\sinh^{2}(gL)\approx 0.25\exp(2gL)$ [36]. The detailed quantum
analysis of OPA in our topological device is shown in Supplementary Section
II. Such a quantum OPA can be used in squeezing light detection [37] and
optical homodyne measurement [38].

###  II.4 Inter-band OPP: entangled biphoton state generation

Furthermore, we can expect the production and manipulation of a frequency
entangled biphoton state derived from the inter-band OPP. Note that all the
pump, signal, and idler modes can couple into the right branch of the diamond-
shaped TPC structure ($f_{s},f_{i}<193$ THz), which is convenient for
extracting broadband entangled photon pairs directly at this branch. We employ
Schmidt decomposition to evaluate the separability of the JSA without
considering part of the phase information [39, 40]. Figure 5(a)-(b)
illustrates the distributions of normalized Schmidt coefficients $\lambda_{n}$
and entanglement entropy $S_{k}$, respectively. It is noteworthy that the
Schmidt coefficients $\lambda_{n}$ signify the probability of acquiring the
$n$th quantum state, with nonzero coefficients (greater than 1) indicating the
frequency entanglement characteristic [39, 41]. Moreover, the entanglement
entropy, denoted as $S_{k}=-\sum\lambda_{n}\log_{2}\lambda_{n}$, and the
Schmidt number, represented by $K=(\sum\lambda_{n}^{2})^{-1}$, are reliable
methods for measuring the degree of entanglement [40]. The entanglement of a
topological quantum state can be verified by $S_{k}>0$ or $K>0$, where a
higher value of $S_{k}$ and $K$ indicates a superior quality of frequency
entanglement. For our topological quantum state, the calculated theoretical
values for the Schmidt number and entropy of entanglement are $K=16.24$ and
$S_{k}=4.42$, respectively, which indicates the emergence of a high-quality
frequency entangled biphoton state in our sandwich TPCs.

Figure 5: Distribution of (a) normalized Schmidt coefficients $\lambda_{n}$
and (b) entanglement entropy $S_{k}$ for the biphoton state generated from
inter-band OPP. (c) Eigenfunctions $\phi_{n}$ ($n=1,2,3,4$) for the biphoton
state. (d) Normalized two-photon spectral distribution with varying pump
frequencies. Figure 6: Field profiles of the FWM process in the diamond-shaped
TPC structure at the frequencies of the (a) signal mode ($f_{s}=196$ THz), (b)
pump mode ($f_{p}=188$ THz), and (c) idler mode($f_{i}=180$ THz),
respectively. Field profiles of the stimulated FWM process in the expanded
diamond-shaped TPC structure at the frequencies of the (a) signal mode
($f_{s}=196$ THz), (b) pump mode ($f_{p}=188$ THz), and (c) idler
mode($f_{i}=180$ THz), respectively.

Due to the symmetry between the signal and idler photons, the eigenfunctions
$\phi$ and $\psi$ in the Schmidt decomposition have the same form. The initial
four eigenfunctions $\phi_{n}$ ($n=1,2,3,4$) are shown in Fig.5(c), which
indicates the orthogonality of each basis function. Also, the number of photon
pairs produced from the FWM process is given by

$\displaystyle S(\omega)$
$\displaystyle=\langle\Psi|a_{s}^{\dagger}(\omega_{s})a_{i}^{\dagger}(\omega_{s})a_{i}(\omega_{s})a_{s}(\omega_{s})|\Psi\rangle$
(4)
$\displaystyle=\frac{\eta^{2}}{c^{2}}\int\mathrm{d}\omega_{\mathrm{s}}\int\mathrm{d}\omega_{\mathrm{i}}|{\mathcal{A}}(\omega_{\mathrm{s}},\omega_{\mathrm{i}})|^{2},$

where $\eta$ is a constant term. Correspondingly, we calculate the normalized
two-photon spectral distribution with pump at different frequencies. As shown
in Fig.5(d), the 3dB bandwidths of the two-photon spectrum are 1.94, 1.54,
1.22, 0.92, and 0.66 THz, respectively, demonstrating the tunability of the
spectral bandwidth. This high-dimensional topological quantum entangled state
with tunable spectral bandwidth enables complex and large-scale quantum
simulations and computations.

Alternatively, the single-photon purity associated with the factorizability of
biphoton states can be implemented through Schmidt decomposition. Purity plays
a crucial role in achieving high-visibility quantum interferences between
photons emanating from the same source. Generally, single-photon purity is
expressed as $\text{Tr}(\hat{\rho}_{\text{s}}^{2})$ where
$\hat{\rho}_{\mathrm{s}}=\mathrm{Tr}_{\mathrm{i}}(|\Psi\rangle\langle\Psi|)$
represents the density operator for the heralded single photon, and
$\mathrm{Tr}{\mathrm{i}}$ is the trace over the idler mode. The heralded
single-photon purity, denoted as $\text{Tr}(\hat{\rho}_{\text{s}}^{2})$, can
be calculated by $\text{Tr}(\hat{\rho}_{\text{s}}^{2})\overset{}{=}K^{-1}$
[42],. Consequently, the single-photon purity for our topological quantum
state is computed as 0.06, corresponding to a highly inseparable quantum
state.

###  II.5 Robustness against disorders for FWM processes

To verify the topological protection of nonlinear FWM, We simulate the FWM
process in the diamond-shaped TPC structure with CW pump excitation [11, 21]
employing COMSOL Multiphysics software. In our numerical model, we use a point
source localized at the input port to excite topological edge modes. Notably,
there is no input for the idler mode, thereby indicating the excitation of
idler modes and the generation of stimulated FWM processes [11]. Here the
frequencies of the pump, signal, and idler modes were chosen as $f_{s}=196$
THz, $f_{p}=188$ THz, and $f_{i}=180$ THz, respectively. As depicted in
Fig.6(a)-(c), the field profiles of topological edge modes at the idler
frequency provide clear evidence of the simulated FWM process. Most
importantly, due to the different frequencies, the pump and signal modes
couple into the right branch of the diamond-shaped structure, while the
generated idler mode couples into the left branch. As a result, photon pairs
are separated, with an idler photon being extracted during the FWM process.

Additionally, we incorporate a diamond-shaped TPC structure to extend our
system’s capacity into a larger spatial domain, enabling enhanced manipulation
of the transmission routes of photonic topological quantum states. As depicted
in Fig.6(d)-(f), within the blue area, a rod’s position is randomly moved by
distances between -0.1a and 0.2a (left branch), and a rod is randomly removed
(right branch). The idler mode generated via FWM processes also exhibits
strong localization along arbitrary topological interfaces. These results
reveal that the topological nature of the QVH effect brings robustness to the
FWM process against sharp bends and defects. These intriguing behaviors enable
the manipulation of the two-photon state’s path and the flexible extraction of
individual photons.

##  III Discussion

###  III.1 Multiple OPPs in honeycomb lattice

Figure 7: Field profiles of the FWM process in the diamond-shaped TPC
structure at the frequencies of the (a) signal mode ($f_{s}=196$ THz), (b)
pump mode ($f_{p}=188$ THz), and (c) idler mode($f_{i}=188$ THz),
respectively. Field profiles of the stimulated FWM process in the expanded
diamond-shaped TPC structure at the frequencies of the (a) signal mode
($f_{s}=196$ THz), (b) pump mode ($f_{p}=188$ THz), and (c) idler
mode($f_{i}=188$ THz), respectively.

We also consider the realization of multifunctional quantum devices in
honeycomb VPCs through multiple OPPs. The valley-Hall honeycomb lattice has
been demonstrated as an effective approach to achieving topologically
protected edge states [43, 44], and there is a potential for realizing
multiple edge modes. The VPCs are composed of a honeycomb lattice featuring
circular holes with a lattice constant of $a=480$ nm. The unperturbed unit
cells ($d_{1}=d_{2}$) demonstrate a $\rm C_{6}$ symmetry, resulting in the
degeneration of $\rm K$ and $\rm K^{\prime}$ valleys in the Brillouin zone.
However, breaking the inversion symmetry ($d_{1}\neq d_{2}$) leads to a
complete bandgap formation near the $\Gamma$ point [43, 44]. The valley Chern
numbers of VPC1 and VPC2 are theoretically calculated as $\rm
C_{K/K^{\prime}}=\pm 1/2$ [43, 44], respectively (Supplementary Section III).

Similarly, we calculate the band structure of a sandwich VPC supercell,
comprising both nontrivial and trivial honeycomb lattices as shown in
Fig.7(a). The dispersion curve reveals the existence of two edge modes
localized within the topological bandgap. Figure 7(b) displays the field
distribution for two valley kink states with $k_{x}=0.4(2\pi/a)$, revealing
that these two modes are localized in two interfaces respectively. We also
simulate the transmission spectra in Fig.7(c), which demonstrates the
frequency division functionality of our diamond-shaped design. Thanks to the
two edge modes for sandwich VPCs, the phase-matching intensity distribution of
FWM processes depicted in Fig.7(d) demonstrates two OPPs, including the intra-
band OPP and inter-band OPP between two edge modes, respectively.
Correspondingly, the JSA of the biphoton state generated from the FWM process
with 190 THz pumping is shown in Fig.7(e). Likewise, there also exist a main
region and two symmetrical bright spots, corresponding to intra-band and
inter-band OPP, respectively. These two OPPs are expected to be performed as
entangled biphoton generation and an OPA process (Supplementary Section III).
Further, we can obtain the JTA of biphotons from the Fourier transformation of
JSA as illustrated in Fig.7(f), leading to a signal-idler correlation with a
10 ps bandwidth. More importantly, we have shown that our topological devices
that conduct multiple OPPs can be implemented in valley-Hall kagome and
honeycomb lattices. Therefore, we believe that multifunctional quantum devices
can be fabricated in a broader range of topological structures in the future.

##  IV Conclusion

In this work, we demonstrate on-chip topological quantum optical devices
capable of performing multiple functions including OPA, frequency separation,
and entangled biphoton generation. We show that there exist two distinct edge
modes corresponding to different frequency ranges in sandwich TPCs. By
employing a diamond structure, we can couple these two edge modes into
separate branches to achieve the separation of spatial modes. Due to the
coexistence of two edge modes, the FWM process enables two types of OPPs,
corresponding to inter-band and intra-band cases, respectively. More
importantly, thanks to distinct transmission paths of the edge modes, these
two OPPs can individually facilitate quantum OPA and the generation of
continuous frequency-entangled photon pairs along separate branches. Moreover,
these quantum processes exhibit topological protection features, demonstrating
robustness against defects and sharp bends. Our proposal offers increased
possibilities for on-chip robust, multifunctional topological quantum devices.

###### Acknowledgements.

This work is supported by the Key-Area Research and Development Program of
Guangdong Province (2018B030325002), the National Natural Science Foundation
of China (62075129, 61975119), the SJTU Pinghu Institute of Intelligent
Optoelectronics (2022SPIOE204), and the Sichuan Provincial Key Laboratory of
Microwave Photonics (2023-04).

Competing interests The authors declare no competing interests.

## References

  * Moody  _et al._ [2020] G. Moody, L. Chang, T. J. Steiner, and J. E. Bowers, Chip-scale nonlinear photonics for quantum light generation, AVS Quantum Science 2 (2020).
  * O’brien [2007] J. L. O’brien, Optical quantum computing, Science 318, 1567 (2007).
  * Caspani  _et al._ [2017] L. Caspani, C. Xiong, B. J. Eggleton, D. Bajoni, M. Liscidini, M. Galli, R. Morandotti, and D. J. Moss, Integrated sources of photon quantum states based on nonlinear optics, Light: Science & Applications 6, e17100 (2017).
  * Sibson  _et al._ [2017] P. Sibson, C. Erven, M. Godfrey, S. Miki, T. Yamashita, M. Fujiwara, M. Sasaki, H. Terai, M. G. Tanner, C. M. Natarajan, _et al._ , Chip-based quantum key distribution, Nature Communications 8, 13984 (2017).
  * Lu  _et al._ [2019] X. Lu, Q. Li, D. A. Westly, G. Moille, A. Singh, V. Anant, and K. Srinivasan, Chip-integrated visible–telecom entangled photon pair source for quantum communication, Nature Physics 15, 373 (2019).
  * Orieux and Diamanti [2016] A. Orieux and E. Diamanti, Recent advances on integrated quantum communications, Journal of Optics 18, 083002 (2016).
  * Pirandola  _et al._ [2018] S. Pirandola, B. R. Bardhan, T. Gehring, C. Weedbrook, and S. Lloyd, Advances in photonic quantum sensing, Nature Photonics 12, 724 (2018).
  * Degen  _et al._ [2017] C. L. Degen, F. Reinhard, and P. Cappellaro, Quantum sensing, Reviews of Modern Physics 89, 035002 (2017).
  * Wang  _et al._ [2021] Y. Wang, K. D. Jöns, and Z. Sun, Integrated photon-pair sources with nonlinear optics, Applied Physics Reviews 8 (2021).
  * Yan  _et al._ [2021] Q. Yan, X. Hu, Y. Fu, C. Lu, C. Fan, Q. Liu, X. Feng, Q. Sun, and Q. Gong, Quantum topological photonics, Advanced Optical Materials 9, 2001739 (2021).
  * You  _et al._ [2023] J. W. You, Z. Lan, Q. Ma, Z. Gao, Y. Yang, F. Gao, M. Xiao, and T. J. Cui, Topological metasurface: From passive toward active and beyond, Photonics Research 11, B65 (2023).
  * Dai  _et al._ [2022a] T. Dai, Y. Ao, J. Bao, J. Mao, Y. Chi, Z. Fu, Y. You, X. Chen, C. Zhai, B. Tang, _et al._ , Topologically protected quantum entanglement emitters, Nature Photonics 16, 248 (2022a).
  * Mittal  _et al._ [2018] S. Mittal, E. A. Goldschmidt, and M. Hafezi, A topological source of quantum light, Nature 561, 502 (2018).
  * Mittal  _et al._ [2021a] S. Mittal, V. V. Orre, E. A. Goldschmidt, and M. Hafezi, Tunable quantum interference using a topological source of indistinguishable photon pairs, Nature Photonics 15, 542 (2021a).
  * Chen  _et al._ [2021] Y. Chen, X.-T. He, Y.-J. Cheng, H.-Y. Qiu, L.-T. Feng, M. Zhang, D.-X. Dai, G.-C. Guo, J.-W. Dong, and X.-F. Ren, Topologically protected valley-dependent quantum photonic circuits, Physical Review Letters 126, 230503 (2021).
  * Jiang  _et al._ [2021] Z. Jiang, Y. Ding, C. Xi, G. He, and C. Jiang, Topological protection of continuous frequency entangled biphoton states, Nanophotonics 10, 4019 (2021).
  * Dai  _et al._ [2022b] T. Dai, Y. Ao, J. Bao, J. Mao, Y. Chi, Z. Fu, Y. You, X. Chen, C. Zhai, B. Tang, _et al._ , Topologically protected quantum entanglement emitters, Nature Photonics 16, 248 (2022b).
  * Afzal  _et al._ [2023] S. Afzal, T. J. Zimmerling, M. Rizvandi, M. Taghavi, T. Hrushevskyi, M. Kaur, V. Van, and S. Barzanjeh, Bright quantum photon sources from a topological floquet resonance, arXiv preprint arXiv:2308.11451 (2023).
  * [19] Z. Jiang, Y. Chen, C. Jiang, and G. He, Generation of quantum optical frequency combs in topological resonators, Advanced Quantum Technologies , 2300354.
  * Jiang  _et al._ [2023a] Z. Jiang, H. Wang, Y. Yang, Y. Shen, B. Ji, Y. Chen, Y. Zhang, L. Sun, Z. Wang, C. Jiang, _et al._ , On-chip topological transport of optical frequency combs in silicon-based valley photonic crystals, arXiv preprint arXiv:2310.15629 (2023a).
  * 21 [2020] Four-wave mixing of topological edge plasmons in graphene metasurfaces, Science Advances 6, eaaz3910 (2020).
  * Smirnova  _et al._ [2019] D. Smirnova, S. Kruk, D. Leykam, E. Melik-Gaykazyan, D.-Y. Choi, and Y. Kivshar, Third-harmonic generation in photonic topological metasurfaces, Physical Review Letters 123, 103901 (2019).
  * Dai  _et al._ [2023] T. Dai, Y. Ao, J. Mao, Y. Yang, Y. Zheng, C. Zhai, Y. Li, J. Yuan, B. Tang, Z. Li, _et al._ , Non-hermitian topological phase transitions controlled by nonlinearity, Nature Physics , 1 (2023).
  * Smirnova  _et al._ [2020] D. Smirnova, D. Leykam, Y. Chong, and Y. Kivshar, Nonlinear topological photonics, Applied Physics Reviews 7 (2020).
  * Jiang  _et al._ [2023b] Z. Jiang, L. Zhou, W. Li, Y. Li, L. Feng, T. Wu, C. Jiang, and G. He, Topological dissipative kerr soliton combs in a valley photonic crystal resonator, Physical Review B 108, 205421 (2023b).
  * Mittal  _et al._ [2021b] S. Mittal, G. Moille, K. Srinivasan, Y. K. Chembo, and M. Hafezi, Topological frequency combs and nested temporal solitons, Nature Physics 17, 1169 (2021b).
  * Vakulenko  _et al._ [2021] A. Vakulenko, S. Kiriushechkina, M. Wang, M. Li, D. Zhirihin, X. Ni, S. Guddala, D. Korobkin, A. Alù, and A. B. Khanikaev, Near-field characterization of higher-order topological photonic states at optical frequencies, Advanced Materials 33, 2004376 (2021).
  * Li  _et al._ [2020] M. Li, D. Zhirihin, M. Gorlach, X. Ni, D. Filonov, A. Slobozhanyuk, A. Alù, and A. B. Khanikaev, Higher-order topological states in photonic kagome crystals with long-range interactions, Nature Photonics 14, 89 (2020).
  * Ezawa [2018] M. Ezawa, Higher-order topological insulators and semimetals on the breathing kagome and pyrochlore lattices, Physical Review Letters 120, 026801 (2018).
  * Mosley  _et al._ [2008] P. J. Mosley, J. S. Lundeen, B. J. Smith, P. Wasylczyk, A. B. U’Ren, C. Silberhorn, and I. A. Walmsley, Heralded generation of ultrafast single photons in pure quantum states, Physical Review Letters 100, 133601 (2008).
  * Cui  _et al._ [2019] C. Cui, R. Arian, S. Guha, N. Peyghambarian, Q. Zhuang, and Z. Zhang, Wave-function engineering for spectrally uncorrelated biphotons in the telecommunication band based on a machine-learning framework, Physical Review Applied 12, 034059 (2019).
  * Cai  _et al._ [2022] W.-H. Cai, Y. Tian, S. Wang, C. You, Q. Zhou, and R.-B. Jin, Optimized design of the lithium niobate for spectrally-pure-state generation at mir wavelengths using metaheuristic algorithm, Advanced Quantum Technologies 5, 2200028 (2022).
  * Riemensberger  _et al._ [2022] J. Riemensberger, N. Kuznetsov, J. Liu, J. He, R. N. Wang, and T. J. Kippenberg, A photonic integrated continuous-travelling-wave parametric amplifier, Nature 612, 56 (2022).
  * Foster  _et al._ [2006] M. A. Foster, A. C. Turner, J. E. Sharping, B. S. Schmidt, M. Lipson, and A. L. Gaeta, Broad-band optical parametric gain on a silicon photonic chip, Nature 441, 960 (2006).
  * Hansryd  _et al._ [2002] J. Hansryd, P. A. Andrekson, M. Westlund, J. Li, and P.-O. Hedekvist, Fiber-based optical parametric amplifiers and their applications, IEEE Journal of Selected Topics in Quantum Electronics 8, 506 (2002).
  * Ledezma  _et al._ [2022] L. Ledezma, R. Sekine, Q. Guo, R. Nehra, S. Jahani, and A. Marandi, Intense optical parametric amplification in dispersion-engineered nanophotonic lithium niobate waveguides, Optica 9, 303 (2022).
  * Nehra  _et al._ [2022] R. Nehra, R. Sekine, L. Ledezma, Q. Guo, R. M. Gray, A. Roy, and A. Marandi, Few-cycle vacuum squeezing in nanophotonics, Science 377, 1333 (2022).
  * Shaked  _et al._ [2018] Y. Shaked, Y. Michael, R. Z. Vered, L. Bello, M. Rosenbluh, and A. Pe’er, Lifting the bandwidth limit of optical homodyne measurement with broadband parametric amplification, Nature communications 9, 609 (2018).
  * Law  _et al._ [2000] C. Law, I. A. Walmsley, and J. Eberly, Continuous frequency entanglement: effective finite hilbert space and entropy control, Physical Review Letters 84, 5304 (2000).
  * Kues  _et al._ [2017] M. Kues, C. Reimer, P. Roztocki, L. R. Cortés, S. Sciara, B. Wetzel, Y. Zhang, A. Cino, S. T. Chu, B. E. Little, _et al._ , On-chip generation of high-dimensional entangled quantum states and their coherent control, Nature 546, 622 (2017).
  * Erhard  _et al._ [2020] M. Erhard, M. Krenn, and A. Zeilinger, Advances in high-dimensional quantum entanglement, Nature Reviews Physics 2, 365 (2020).
  * Vicent  _et al._ [2010] L. E. Vicent, A. B. U’Ren, R. Rangarajan, C. I. Osorio, J. P. Torres, L. Zhang, and I. A. Walmsley, Design of bright, fiber-coupled and fully factorable photon pair sources, New Journal of Physics 12, 093027 (2010).
  * Yang  _et al._ [2020] Y. Yang, Y. Yamagami, X. Yu, P. Pitchappa, J. Webber, B. Zhang, M. Fujita, T. Nagatsuma, and R. Singh, Terahertz topological photonics for on-chip communication, Nature Photonics 14, 446 (2020).
  * Shalaev  _et al._ [2019] M. I. Shalaev, W. Walasik, A. Tsukernik, Y. Xu, and N. M. Litchinitser, Robust topologically protected transport in photonic crystals at telecommunication wavelengths, Nature Nanotechnology 14, 31 (2019).