# Photon Conversion and Interaction on Chip

Jia-Yang Chen  These authors contributed equally  Zhan Li  These authors
contributed equally  Zhaohui Ma  Chao Tang  Heng Fan  Yong Meng Sua  Yu-Ping
Huang  Electronic address [1]<EMAIL_ADDRESS>
Electronic address [2]<EMAIL_ADDRESS>Department of Physics, Stevens
Institute of Technology, 1 Castle Point Terrace, Hoboken, New Jersey, 07030,
USA  Center for Quantum Science and Engineering, Stevens Institute of
Technology, 1 Castle Point Terrace, Hoboken, New Jersey, 07030, USA

###### Abstract

The conversion and interaction between quantum signals at a single-photon
level are essential for scalable quantum photonic information technology.
Using a fully-optimized, periodically-poled lithium niobate microring, we
demonstrate ultra-efficient sum-frequency generation on chip. The external
quantum efficiency reaches $(65\pm 3)\%$ with only $(104\pm 4)$ $\mu$W pump
power, improving the state-of-the-art by over one order of magnitude. At the
peak conversion, $3\times 10^{-5}$ noise photon is created during the cavity
lifetime, which meets the requirement of quantum applications using single-
photon pulses. Using pump and signal in single-photon coherent states, we
directly measure the conversion probability produced by a single pump photon
to be $10^{-5}$—breaking the record by 100 times—and the photon-photon
coupling strength to be 9.1 MHz. Our results mark a new milestone toward
quantum nonlinear optics at the ultimate single photon limit, creating new
background in highly integrated photonics and quantum optical computing.

Unlike electrons, atoms, or any other material particles, photons do not
interact with each other in vacuum. Even when mixed in optical media of the
best known nonlinearities, their interaction is so weak that high optical
intensities are needed to produce an appreciable effect. This inefficiency
accounts for significant difficulties facing practical implementations of
quantum transduction [1], faithful entanglement swapping [2, 3], and heralded
entanglement generation [4, 5], to name a few. It also prohibits the
construction of nonlinear photon-photon gates, thus forming a bottleneck for
the development of scalable quantum computers at room temperature [6].

The recent advances in nanophotonics bring hopes to overcome this challenge,
by providing tight optical confinement, strong mode overlap, and extended
interaction length. Encouragingly, optical processes with improved efficiency
have now been demonstrated in nanophotonic circuits made of silicon nitride
[7, 8], aluminum nitride [9], gallium arsenide [10], gallium phosphide [11],
aluminium gallium arsenide[12] and lithium niobate [13, 14]. Among various
candidates, thin-film lithium niobate (TFLN) on insulator has quickly arisen
to a material platform of choice, due to its favorable ferro-electricity, wide
optical transparency window, strong second-order nonlinearity $\chi^{(2)}$,
and outstanding electro-optical responses. Compared with third-order nonlinear
($\chi^{(3)}$) materials, TFLN is centro-asymmetric and possesses
exceptionally large second-order nonlinearity ($\chi^{(2)}$) to produce orders
of magnitude stronger effects, as desirable to optical nonlinearities at a
single photon level. Thus far, a variety of TFLN microresonators have been
demonstrated with impressive nonlinearities [15, 16, 17, 13, 14, 18]. However,
their performance has been capped by the use of relatively small $\chi^{(2)}$
susceptibilities (e.g., $d_{31}\sim$ 4.7 pm/V) [15], poor mode overlapping
[16, 18], and/or low photon-extraction efficiency [15, 16, 17, 13]. For photon
conversion and interaction, while their high efficiency has been predicted
from, e.g., second-harmonic generation (SHG) [15, 16, 17] or parametric
downconversion experiments [13], there has been no direct demonstration.

Here, we present a TFLN resonator that overcomes all aforementioned
shortcomings and delivers its promised high efficiency for photon conversion
and interaction. As illustrated in Fig.1(a), it is an overly coupled, triply
resonant, periodically poled lithium niobate (PPLN) microring resonator for
sum-frequency generation (SFG). All interacting light waves are in the low-
loss fundamental modes with nearly perfect overlap and interact through TFLN’s
largest $\chi^{(2)}$ susceptibility tensor element (e.g., $d_{33}\sim$ 27
pm/V). It achieves an impressive photon-photon coupling strength of $g$ = 9.1
MHz (angular frequency). Crucially, by strongly overcoupling the cavity to
minimize the extraction loss of the sum-frequency (SF) photons, we demonstrate
photon conversion at a record-high external efficiency of $65\%$ with only
about 100 $\mu$W pump power, marking orders of magnitude improvement over the
state of the art across all existing photonic platforms; see Table 1. At the
peak conversion, the on-chip noise photon flux is only $3\times 10^{-5}$
photons per 100-ps cavity lifetime, despite small-detuning pumping. This
ultrahigh external efficiency yet low noise create new opportunities in
various applications like quantum frequency conversion, optical squeezing, and
phase sensitive amplification.

Material Platform | Structure | Nonlinear Process |  $Q_{l}$ ($\times 10^{5}$) | $\eta_{\mathrm{con}}$ |  Pump Power | Ref.
---|---|---|---|---|---|---
Si3N4 | microring | FWM-BS | 1.5/1.5/2.4 | 60% | 50 & 8 mW | [7]
Si3N4 | microring | DFWM | 2.8/3.0/1.2 | 13% | 0.33 mW | [8]
AlN | microring | SFG | 3.0/3.0/1.4 | 42% | 35 mW | [9]
PPLN | microring | SHG | 1.3/3.0 | 10% | 0.3 mW | [13]
PPLN | microring | SHG | 1.5/0.6 | 26% | 1.8 mW | [14]
PPLN | millimeter disk | SHG | 120/80 | 18% | 9 mW | [19]
PPLN | microring | SFG | 1.3/1.3/2.6 | 65% | 0.1 mW | this work
Table 1: The state-of-the-art frequency conversion in $\chi^{(2)}$ and
$\chi^{(3)}$ cavities. $\eta_{\mathrm{con}}$: conversion efficiency by photon
number; FWM-BS: four-wave mixing Bragg scattering; DFWM: degenerate four-wave
mixing. $Q_{l}$ lists the loaded quality factors for the pump, signal and SF
waves in the case of FWM-BS, DFWM, and SFG, and the pump and second-harmonic
waves in the case of SHG. This table only includes those whose conversion
efficiency is over $10\%$. Note: a recent arxiv preprint reported SHG of
$\eta_{\mathrm{con}}=33\%$ [18]. According to its reported normalized
efficiency of $602\%$/mW, the required pump power shall be
$P_{p}=16~{}\frac{66\%}{602\%/\mathrm{mW}}=1.75~{}\mathrm{mW}$ [15, 14], which
is over 16 times larger than its claimed value. We have not included it in
this table due to this apparent inconsistence. Figure 1: Integrated TFLN
circuits for photon conversion and interaction. (a) Schematic of the Z-cut
periodically poled microring, where the pump of $\omega_{p}$ and signal of
$\omega_{s}$ couple into the microring and generate the SF light of
$\omega_{f}$ via a $\chi^{(2)}$ process. A pulley coupler is designed for over
coupling all light waves, for high photon extraction efficiency. Insets
illustrate triply resonant and quasi-phase matching conditions. (b) and (c)
are the SEM images of the etched pulley coupler before poling process and the
periodic poled microring after removing the poling electrodes, respectively.

Beside the large coupling strength, our device’s high cavity quality for all
interacting waves provides extended interaction length to boost optical
nonlinearities towards the single photon regime. To assess this prospect, we
further perform photon interaction between two single-photon signals in weak
coherent states. Our measurement directly shows that with only one pump photon
in the microring, the external quantum efficiency for signal photon conversion
is $\sim 10^{-5}$, while the best efficiency reported hitherto is $10^{-7}$
[20]. The internal Rabi oscillation angle produced by the pump photon is 0.01,
which can be improved to approach unity by further reducing the cavity loss.

As a whole, our results constitute a new milestone along the long pursuit of
nonlinear optics in its ultimate quantum limit, where a single photon is
enough to produce significant nonlinear effects. While there is still a good
journey to take before hitting the finish line, the fact that we can attain
the theoretical performance of this device is critical for us to take steps
forward. By further improving the cavity quality, strong photon-photon
interaction is within sight. Based on the current nonlinear parameters, a
cavity qualify factor of $10^{8}$—which has been demonstrated in lithium
niobate microdisks [18]—will enable C-NOT gate between single photons.
Meanwhile, the demonstrated photon conversion and interaction efficiency can
already elevate the performance of nonlinear-optical devices for heralded
entanglement generation, faithful entanglement swapping, and so on.

Device design: In $\chi^{(2)}$ cavity, the effective Hamiltonian describing
SFG between photons in their single modes is

$\hat{H}_{\textrm{eff}}=\hbar\mathnormal{g}(\hat{a}_{p}\hat{a}_{s}\hat{a}_{f}^{\dagger}+\hat{a}_{p}^{\dagger}\hat{a}_{s}^{\dagger}\hat{a}_{f}),\\\
$ (1)

where $\\{\hat{a}_{j}\\}$ are the annihilation operators with $j=p,s,f$
standing for the pump, signal and SF light, respectively, each with angular
frequency $\omega_{j}$. ${g}$ is the photon-photon coupling strength, which
can be interpreted as the effective Rabi frequency produced by a pump photon
(see Supplementary Material.1). It is given by

$g\propto\frac{d_{\textrm{eff}}\xi}{\sqrt{V_{\textrm{eff}}}}\times\delta(m_{f}-m_{p}-m_{s}-M)$
(2)

where $d_{\textrm{eff}}$ is the effective nonlinear susceptibility. $\xi$ is
the mode overlapping factor. $V_{\textrm{eff}}$ is the effective mode volume.
$m_{j}$ is the azimuthal order of the cavity modes, and $M$ is the azimuthal
poling grating number, so that $\delta(m_{f}-m_{p}-m_{s}-M)$ accounts for
quasi-phase matching (QPM) by periodic poling.

In this work, we use a microring with a radius of 80 $\mu$m and a cross-
section of 600 nm in height and 1700 nm in top-width. The pump and signal are
both in the telecom C-band and their SF is in the visible band, chosen so with
repeater-based quantum communications in mind. To maximize $g$, all three
waves are in the fundamental quasi-transverse-magnetic (quasi-TM) modes and
interact through TFLN’s largest nonlinear tensor $d_{33}$ with over 90% mode
overlap. As shown in Fig.1, concentric periodic poling is applied to the
microring for QPM. To ensure triple resonances for all waves, fine temperature
tuning ($\sim$ 0.01 ∘C) is applied to compensate for any resonant mismatch due
to any fabrication error.

For photon conversion and interaction, the device figure of merit is the
external quantum efficiency (QE) defined as

$\eta_{\mathrm{QE}}=\frac{N_{f}}{N_{s}},$ (3)

where $N_{s}$ is the number of input signal photons to the cavity and $N_{f}$
is that of converted SF photons at the cavity output (thus accounting for any
internal cavity loss). This is in contrast to previous demonstrations, where
critical coupling was adopted to maximize the intracavity optical power for
high conversion efficiency [15, 16, 17, 13]. However, most of the input power
and about half of the converted photons are lost inside the cavity, rendering
a rather low QE while prohibiting cascaded operations. For practical
applications, one instead needs to over-couple the cavity so that the photons
can be extracted out before significantly lost in the cavity.

Under QPM and triple resonance (see Supplementary Material.1), the maximum QE
is given by [21]:

$\displaystyle\eta_{\mathrm{QE}}^{\mathrm{max}}\approx\frac{Q_{s,l}}{Q_{s,c}}\frac{Q_{f,l}}{Q_{f,c}},$
(4)

where $Q_{j,o}$ is the quality factor with $o=c,l$ denoting the coupling and
loaded Q, respectively. It is reached with an optimal pump power

$\displaystyle P_{p}^{\mathrm{opt}}\approx
8~{}\frac{\eta_{\mathrm{QE}}^{\mathrm{max}}}{\eta_{\mathrm{tran}}^{\mathrm{nor}}},$
(5)

where $\eta_{\mathrm{tran}}^{\mathrm{nor}}=P_{f}/(P_{s}P_{p})$ is the
normalized power transduction efficiency with $P_{j}$ the optical power of the
$j$-th wave. In our case, $\eta_{\mathrm{QE}}^{\mathrm{max}}\approx 65\%$ and
$\eta_{\mathrm{tran}}^{\mathrm{nor}}\approx 4.5\%/\mu$W, so that the optimal
power is around 115 $\mu$W.

We use a pulley coupler in optimized dimensions to achieve proper over-
coupling for both the signal and SF modes. This is done by first determining
its top width by requiring $n_{p}R_{p}=n_{r}R_{r}$ [22], where $n_{p,r}$ are
the effective refractive indices of the pulley waveguide and microring modes
for the sum-frequency wave, and $R_{p}$ and $R_{r}$ denote their radii. For
the SF wave, the waveguide-microring coupling strength is proportional to the
length of the pulley coupler, while inversely proportional to their gap. For
the signal mode, in contrast, the coupling strength varies as
$\mathrm{sinc}(\Delta\Phi)$, where $\Delta\Phi$ is the coupling phase
mismatch. This allows to carefully design the dimensions to create the
desirable over-coupling for both signal and sum-frequency waves.

Figure 2: (a) Optical spectra of the interacting TM00 cavity modes at (i)
1560.15 nm, (ii) 1551.85 nm, and (iii) 778.00 nm, respectively. (b)
Illustration of possible parametric conversion processes in the microring. (c)
Original optical spectrum before any optical filtering when strong pump and
weak signal are applied. In high conversion region, signal starts to be
depleted. Insert gives the zoom-in spectrum around the SF band. After
correcting for the coupling loss, the on-chip power of pump, signal and SF
waves are -13.8 dBm, -44.3 dBm and -44.0 dBm, respectively.

Device characterization: The details of our device fabrication and experiment
setup are presented in Method and Supplementary .2. The fiber-chip-fiber
coupling losses are measured to be 8 $\pm$ 0.15 dB at 1556 nm and 9.5 $\pm$
0.2 dB at 778 nm, respectively. To find phase matching, we first search for
strong SHG across the C-band by sweeping an infrared laser while optimizing
the chip’s temperature. This gives several cavity modes around 1556 nm, based
on which we select a set of cavity modes at 1560.15 nm, 1551.85 nm, and 778.00
nm, considering the over-coupling requirement and limited by our visible band-
pass filter (BPF, bandwidth $\sim$3 nm, 760 to 780 nm). To reduce the Raman
background, we designate the 1560.15 nm mode to the pump. For this set,
$m_{p}=602$, $m_{s}=606$, $m_{f}=1357$, so that $M=149$. The loaded quality
factors $Q_{j,l}$ are measured for each modes, while the coupling $Q_{j,c}$
and intrinsic $Q_{j,0}$ factors are calculated by fitting the resonance
spectra, as shown in Fig. 2 (a). Those Q’s, according to Eq. (2), give the
highest possibl external quantum efficiency of
$\eta_{\mathrm{QE}}^{\mathrm{max}}\approx 65\%$. For an even higher
efficiency, the cavity needs to be further overcoupled.

Low-noise Frequency Conversion:  We perform SFG using the setup detailed in
Fig. S1. We first couple a strong pump at 1560 nm and a weak signal at 1552 nm
into the microring, and fine tune the microring temperature and the laser
wavelengths to verify QPM and triple resonance. The resulting spectrum,
measured without any filtering thus manifesting all possible nonlinear
processes, is shown in Fig. 2(c). It exhibits a clean profile with low
baseline, except for a residual second-harmonic peak at $\sim$780 nm by the
strong pump. This peak is significant only in the high conversion regime
(i.e.,$>$50$\%$) and can be conveniently filtered out. In this experiment, we
reject it using a $\sim$3 nm bandpass filter centered at 778 nm. The otherwise
low background over the entire spectral range shows that all other competing
processes are well suppressed, as desirable for quantum applications.

In this experiment, the signal power is fixed to be about 37 nW on chip, while
the pump power is gradually increased. During the measurement, we only need to
slightly optimize the temperature within 45 $\pm$ 0.3 ∘C and tune the
wavelengths of both lasers within $\pm$ 20 pm to compensate for slight phase
mismatch and resonance drift caused by thermo-optical and photorefractive
effects [14]. The quantum efficiency as a function of the on-chip pump power
is shown in Fig. 3. Thanks to the over-coupling and nearly ideal poling,
$(65\pm 3)\%$ quantum efficiency is achieved with at $(104\pm 4)$ $\mu$W pump
power. Taking into account all insertion losses both on and off chip, this
corresponds to about $9\%$ total efficiency. In the low conversion region, the
normalized power transduction efficiency is fitted to be about 4.5$\%$
$\mu$W-1. By fitting the experimental data with the steady-state solution to
Eqs.(S1-S3), as shown in Fig. 3, the photon-photon coupling coefficient $g$ is
determined to be 8.2 MHz.

The above results speaks to the ultrahigh efficiency. For quantum frequency
conversion, it is critical that no significant in-band noise is injected
during the conversion. As illustrated in Fig. 2(b), there are multiple
processes that can produce in-band noise, such as Raman scattering from strong
classical pump to the signal band followed by SFG, Raman scattering by the
pump’s residue SH light, and spontaneous parametric down-conversion (SPDC)
followed by SHG and SFG. To quantify their total contributions, we measure the
noise photon flux generated in the SF band when only the pump is applied. To
ensure total rejection of any out-band noise, the SF photons are passed
through additional free-space filters (see Supplementary Material .2), before
detected by a silicon-based single-photon detector (Si-SPD, quantum
efficiency: 50$\%$, dark count: 250 Hz). The results are plotted in Fig. 3,
where the increase of the noise photon flux is between quadratic and cubic
with the pump power. This result indicates that besides Raman scattering, the
cascaded SPDC and SFG process also presents, similar to what we observed in a
PPLN nanowaveguide [23]. At the $65\%$ peak QE, the on-chip noise photon flux
is 0.3 MHz under continuous-wave pumping. If using $\sim$100 ps pulses that
match the cavity lifetime, the noise photon per pulse is $3\times 10^{-5}$,
which is low especially given the small detuning between the pump and signal
that are both in the telecom C-band. This noise level can be substantially
lowered by, for example, further detuning the pump from the signal [24].

Figure 3: SFG efficiency $\eta_{\mathrm{QE}}$ (left-axis, red square) and the
generated noise photon flux $N_{\mathrm{noise}}$(right-axis, black square)
plotted against the on-chip pump power. $\eta_{\mathrm{QE}}$ = 65$\%$ is
obtained at the pump power around 100 $\mu$W. Solid red curve represents the
prediction of the coupled mode equations (see Supplementary Material.1) using
the actual parameters of the microring with only one fitting parameter: $g$ =
8.2 MHz. Solid black line is fitted to the pump power $P$ as
$N_{\mathrm{noise}}\sim P^{2.4}$. The error bars in left-axis and x-axis are
estimated according to coupling instability. The error bar of noise photon in
right-axis is estimated by uncertainty assuming Poissonian photon counting
statistics. Figure 4: Quantum efficiency $\eta_{\mathrm{QE}}$ versus the
intracavity mean pump photon number. Detector dark counts of 250 Hz have been
subtracted, and the coupling loss of 4.75 dB and detector efficiency of 50$\%$
have been accounted for. Inset is an zoom-in to shown a quantum effciency of
$\eta_{\mathrm{QE}}=10^{-5}$ is achieved at one intracavity pump photon
(corresponding to photon flux $N_{p}=2.8$ GHz), where the intracavity signal
photon number is fixed at 0.25 (corresponding to photon flux $N_{s}=0.7$ GHz).
Solid red line represents the simulated results using the actual parameters of
the microring with only one free parameter $g$ = 9.1 MHz, fitted with the
coupled-mode equations (see Supplementary Material.1). All error bars are
estimated assuming Poissonian photon counting statistics. Figure 5:
Illustration of a TFLN integrated photonic circuit for heralded entanglement
generation.

Interaction Between Photon-level Coherent States: Strong interaction between
single photons is of great values for both fundamental optics studies and
quantum applications such as faithful quantum entanglement swapping [2, 3],
heralded entanglement source [4, 5], and device-independent quantum key
distribution [25]. Here, we characterize the device responses in the single
photon regime by attenuating the pump and signal to weak coherent states at
single photon levels. Their photon fluxes are carefully monitored by two
superconducting single-photon detectors (SNSPDs, see Supplementary Material
.2).

To maximize the detection efficiency for the SF photons, we remove the free-
space filtering system—used in the last section to reject background noise
created by the strong pump—and directly couple the SF photons from the chip to
the Si-SPD via a lensed fiber. To ensure that the pump photons do not generate
background counts (that could contribute to the over-estimation of the single-
photon nonlinearity), we block the signal and verify that even at the highest
on-chip flux ($\sim$ 5 GHz), the photon counts remain at its dark count level
($\sim$ 250 Hz). Here, the total detection efficiency for the sum-frequency
photons is $\eta=\eta_{d}\eta_{c}\approx 17\%$, with detector efficiency
$\eta_{d}\sim 50\%$ and coupling efficiency $\eta_{c}\sim 33.5\%$.

The measurement results are shown in Fig. 4, where we observe linear
dependency of the quantum efficiency on the intracavity pump photon number.
The on-chip quantum efficiency is about $10^{-5}$ when there is one pump
photon on average in the cavity, after correcting for the coupling loss and
finite detector efficiency. By fitting with the coupling-mode equations, we
extract the photon-photon coupling coefficient $g$ to be 9.1 MHz, compared to
8.2 MHz as from the classical measurement. This slightly higher value may come
from better QPM or triple resonance for single-photon light waves, due to the
absence of thermo-optical and photorefrative effects.

The above quantum efficiency gives the probability of a signal photon at the
cavity input being converted to its SF and appear at the cavity output. It has
accounted for the signal coupling loss and the SF extraction loss, thus
constituting a direct measure of the device figure of merit that dictates its
performance in quantum applications. Another measure, of less practical
relevance but nonetheless describing an intrinsic property, is the internal
Rabi rotation angle $\theta=2gQ_{p,i}/\omega_{p}$. That’s, Eq. (1) can be
interpreted as the pump induced Rabi oscillation between signal and SF
photons. $\theta$ then gives how much Rabi rotation can a pump photon induce
during it is lost in the cavity. In our case, $g=9.1\times 10^{6}$ and
$Q_{p,i}=7.1\times 10^{5}$, so that $\theta=0.01$, which is already
appreciable from a fundamental standpoint. Increasing $\theta$ to $\pi/2$
would require improving $Q_{p,i}$ to $10^{8}$, which has been demonstrated in
polished TFLN microdisks [18].

Heralded Entanglement Generation: The demonstrated photon-level nonlinearity
implies unprecedented performance in quantum information science and
technology. In particular, because of lithium niobate’s exceptional optical
properties in multiple aspects, the demonstrated photon conversion and
interaction are ready to be integrated with other passive and active elements
on the same chip, such as PPLN wavegudies [26, 27], electro-optical modulators
[28], frequency comb sources [29, 30, 31, 32], and microring filters [33], to
create functional quantum devices of practical impacts [34]. Compared with the
existing table-top or assembled systems, the detrimental insertion loss will
be eliminated, and the mechanical and optical stabilities are expected to be
exceptional.

As an example, in Fig. 5 we present the design of an integrated chip for
heralded entanglement generation, following the scheme in [2]. It consists of
a PPLN nanowaveguide to create photon pairs by SPDC, a series of thermally-
tuned microrings for optical filtering, directional couplers for photon
combination, and a PPLN microring for photon conversion. The process starts
with passing a visible pump through a PPLN nanowaveguide to generate via SPDC
photon pairs simultaneously over multiple wavelength channels,
$\omega_{s1}+\omega_{i1}$, $\omega_{s2}+\omega_{i2}$,…, and
$\omega_{sn}+\omega_{in}$ [26, 27]. The signal and idler photons will each be
picked and separated into different arms by using coupled add-drop filters for
high extinction while eliminating free spectrum ambiguity. In each arm,
amorphous-silicon over-cladding will be applied to further reject any residue
pump in the visible band. The signal photons will be recombined via cascaded
directional couplers and sent into a PPLN microring, where they are interact
to create a SF photon. The existence of a photon pair in the idler channels
will be heralded upon the detection of the SF photon. To create entanglement
in time bins, the SF photon needs to pass through a Franson interferometer and
be detected in a superposition time-bin state [2]. Unlike schemes using SPDC
and linear optical Bell state measurement, where the success probability is
fundamentally capped at 50% [35], this scheme is deterministic in that upon
heralding, the entangled photon pair exists with nearly certainty. In this
design, with 12 pm net filtering bandwidth and 100 ps pump pulses, one can
drive the SPDC at 1% photon pair production rate per pulse for each channel.
The heralding rate can approach 10 Hz for the demonstrated $10^{-5}$ photon
conversion, which would correspond to orders of magnitude improvement over
previous demonstrations [4, 5]·

Discussion: We have demonstrated photon conversion and interaction with record
high efficiency and low noise in a Z-cut, periodically poled microring on
thin-film lithium niobate. Combining nearly perfect QPM, tight mode
confinement, high cavity quality, and efficient photon extraction, we achieved
65% wavelength transduction with only about 100 microwatt pump power,
advancing the state of the art by large. Despite a small detuning between the
signal and pump, at the peak conversion only $3\times 10^{-5}$ noise photons
are created over the cavity lifetime, thanks to the deep single-mode condition
and suppression of side processes. The same device allowed nonlinear
interaction between two single-photon level coherent states, where the photon-
photon coupling strength reaches 9.1 MHz, and a single photon can produce 0.01
internal Rabi rotation angle. The external quantum efficiency is directly
measured to be about $10^{-5}$, compared with the best reported efficiency of
$10^{-7}$. Our results mark new milestones towards quantum nonlinear optics in
the single photon regime, with broad implications in areas of fundamental
studies and applied quantum information technology. Combining with other
favorable optical proprieties of thin film lithium niobate, a superior
platform of photonic integrated circuits is within sight for scalable quantum
applications.

## Methods

The entire device is fabricated on a magnesium-doped Z-cut LNOI wafer (NANOLN
Inc.), with a 600-nm thick LN thin film bonded on 2 $\mu$m silicon dioxide
layer above a silicon substrate. First, the microring and waveguide structure
are defined using hydrogen silsesquioxane (HSQ, Fox-16) by electron beam
lithography. The top width and the radius of the microring are 1.7 $\mu$m and
80 $\mu$m, respectively. Then, ICP Argon milling is applied to shallowly etch
the structures, where 430-nm thick LN is etched with 170-nm LN remaining, and
the sidewall angle is approximately 64∘. The optimized pulley coupler, shown
in Fig. 1(b) with the pulley top width of $w_{pulley}$ = 400 nm, the gap of
$g_{pulley}$ = 650 nm, and the length of $L_{pulley}$ = 40 $\mu$m, is created
to increase the ring-bus waveguide coupling to attain simultaneously over-
coupling condition for both the visible and IR lightwaves. Then, a concentric
periodically-poled region with period of $\Lambda=3.37$ $\mu$m given by
$\Lambda=2\pi R/M$, M = 149 and near 50$\%$ duty cycle (see Fig. 1(c)) is
created via several 1-ms and 450-V electrical pulses using a similar process
described in [27]. After removing the poling electrodes, the chip is coated
with 1.5 $\mu$m silicon dioxide. Finally the chip is diced and polished.

## References

  * [1] N. Maring, D. Lago-Rivera, A. Lenhard, G. Heinze, and H. de Riedmatten, “Quantum frequency conversion of memory-compatible single photons from 606 nm to the telecom c-band,” Optica 5, 507–513 (2018).
  * [2] N. Sangouard, B. Sanguinetti, N. Curtz, N. Gisin, R. Thew, and H. Zbinden, “Faithful entanglement swapping based on sum-frequency generation,” Physical review letters 106, 120403 (2011).
  * [3] Y. Li, Y. Huang, T. Xiang, Y. Nie, M. Sang, L. Yuan, and X. Chen, “Multiuser time-energy entanglement swapping based on dense wavelength division multiplexed and sum-frequency generation,” Physical review letters 123, 250505 (2019).
  * [4] C. Wagenknecht, C.-M. Li, A. Reingruber, X.-H. Bao, A. Goebel, Y.-A. Chen, Q. Zhang, K. Chen, and J.-W. Pan, “Experimental demonstration of a heralded entanglement source,” Nature Photonics 4, 549–552 (2010).
  * [5] S. Barz, G. Cronenberg, A. Zeilinger, and P. Walther, “Heralded generation of entangled photon pairs,” Nature photonics 4, 553–556 (2010).
  * [6] D. E. Chang, V. Vuletić, and M. D. Lukin, “Quantum nonlinear optics—photon by photon,” Nature Photonics 8, 685–694 (2014).
  * [7] Q. Li, M. Davanço, and K. Srinivasan, “Efficient and low-noise single-photon-level frequency conversion interfaces using silicon nanophotonics,” Nature Photonics 10, 406–414 (2016).
  * [8] X. Lu, G. Moille, Q. Li, D. A. Westly, A. Singh, A. Rao, S.-P. Yu, T. C. Briles, S. B. Papp, and K. Srinivasan, “Efficient telecom-to-visible spectral translation through ultralow power nonlinear nanophotonics,” Nature Photonics 13, 593–601 (2019).
  * [9] J.-Q. Wang, Y.-H. Yang, M. Li, X.-X. Hu, J. B. Surya, X.-B. Xu, C.-H. Dong, G.-C. Guo, H. X. Tang, and C.-L. Zou, “Efficient frequency conversion in a degenerate $\chi$ (2) microresonator,” Physical Review Letters 126, 133601 (2021).
  * [10] L. Chang, A. Boes, P. Pintus, J. D. Peters, M. Kennedy, X.-W. Guo, N. Volet, S.-P. Yu, S. B. Papp, and J. E. Bowers, “Strong frequency conversion in heterogeneously integrated gaas resonators,” APL Photonics 4, 036103 (2019).
  * [11] D. J. Wilson, K. Schneider, S. Hönl, M. Anderson, Y. Baumgartner, L. Czornomaz, T. J. Kippenberg, and P. Seidler, “Integrated gallium phosphide nonlinear photonics,” Nature Photonics 14, 57–62 (2020).
  * [12] L. Chang, W. Xie, H. Shu, Q.-F. Yang, B. Shen, A. Boes, J. D. Peters, W. Jin, C. Xiang, S. Liu _et al._ , “Ultra-efficient frequency comb generation in algaas-on-insulator microresonators,” Nature communications 11, 1–8 (2020).
  * [13] Z. Ma, J.-Y. Chen, Z. Li, C. Tang, Y. M. Sua, H. Fan, and Y.-P. Huang, “Ultrabright quantum photon sources on chip,” Physical Review Letters 125, 263602 (2020).
  * [14] J.-Y. Chen, C. Tang, M. Jin, Z. Li, Z. Ma, H. Fan, S. Kumar, Y. M. Sua, and Y.-P. Huang, “Efficient frequency doubling with active stabilization on chip,” (2021).
  * [15] J. Lu, J. B. Surya, X. Liu, A. W. Bruch, Z. Gong, Y. Xu, and H. X. Tang, “Periodically poled thin-film lithium niobate microring resonators with a second-harmonic generation efficiency of 250,000%/w,” Optica 6, 1455–1460 (2019).
  * [16] J.-Y. Chen, Z.-H. Ma, Y. M. Sua, Z. Li, C. Tang, and Y.-P. Huang, “Ultra-efficient frequency conversion in quasi-phase-matched lithium niobate microrings,” Optica 6, 1244–1245 (2019).
  * [17] J. Lu, M. Li, C.-L. Zou, A. A. Sayem, and H. X. Tang, “Toward 1% single-photon anharmonicity with periodically poled lithium niobate microring resonators,” Optica 7, 1654–1659 (2020).
  * [18] R. Gao, H. Zhang, F. Bo, W. Fang, Z. Hao, N. Yao, J. Lin, J. Guan, L. Deng, M. Wang, L. Qiao, and Y. Cheng, “Broadband highly efficient nonlinear optical processes in on-chip integrated lithium niobate microdisk resonators of q-factor above 108,” (2021).
  * [19] V. S. Ilchenko, A. A. Savchenkov, A. B. Matsko, and L. Maleki, “Nonlinear optics and crystalline whispering gallery mode cavities,” Physical review letters 92, 043903 (2004).
  * [20] Y. Li, T. Xiang, Y. Nie, M. Sang, and X. Chen, “Nonlinear interaction between broadband single-photon-level coherent states,” Photonics Research 5, 324–328 (2017).
  * [21] I. Breunig, “Three-wave mixing in whispering gallery resonators,” Laser & Photonics Reviews 10, 569–587 (2016).
  * [22] E. S. Hosseini, S. Yegnanarayanan, A. H. Atabaki, M. Soltani, and A. Adibi, “Systematic design and fabrication of high-q single-mode pulley-coupled planar silicon nitride microdisk resonators at visible wavelengths,” Optics express 18, 2127–2136 (2010).
  * [23] H. Fan, Z. Ma, J. Chen, Z. Li, C. Tang, Y. Sua, and Y. Huang, “Photon conversion in thin-film lithium niobate nanowaveguides: A noise analysis,” arXiv preprint arXiv:2102.07044 (2021).
  * [24] A. Singh, Q. Li, S. Liu, Y. Yu, X. Lu, C. Schneider, S. Höfling, J. Lawall, V. Verma, R. Mirin _et al._ , “Quantum frequency conversion of a quantum dot single-photon source on a nanophotonic chip,” Optica 6, 563–569 (2019).
  * [25] U. Vazirani and T. Vidick, “Fully device independent quantum key distribution,” Communications of the ACM 62, 133–133 (2019).
  * [26] J. yang Chen, Y. M. Sua, Z. hui Ma, C. Tang, Z. Li, and Y. ping Huang, “Efficient parametric frequency conversion in lithium niobate nanophotonic chips,” OSA Continuum 2, 2914–2924 (2019).
  * [27] J.-Y. Chen, C. Tang, Z.-H. Ma, Z. Li, Y. M. Sua, and Y.-P. Huang, “Efficient and highly tunable second-harmonic generation in z-cut periodically poled lithium niobate nanowaveguides,” Optics Letters 45, 3789–3792 (2020).
  * [28] M. Jin, J.-Y. Chen, Y. M. Sua, and Y.-P. Huang, “High-extinction electro-optic modulation on lithium niobate thin film,” Optics letters 44, 1265–1268 (2019).
  * [29] Y. He, Q.-F. Yang, J. Ling, R. Luo, H. Liang, M. Li, B. Shen, H. Wang, K. Vahala, and Q. Lin, “Self-starting bi-chromatic linbo 3 soliton microcomb,” Optica 6, 1138–1144 (2019).
  * [30] C. Wang, M. Zhang, M. Yu, R. Zhu, H. Hu, and M. Loncar, “Monolithic lithium niobate photonic circuits for kerr frequency comb generation and modulation,” Nature communications 10, 1–6 (2019).
  * [31] M. Zhang, B. Buscaino, C. Wang, A. Shams-Ansari, C. Reimer, R. Zhu, J. M. Kahn, and M. Lončar, “Broadband electro-optic frequency comb generation in a lithium niobate microring resonator,” Nature 568, 373–377 (2019).
  * [32] Z. Gong, X. Liu, Y. Xu, and H. X. Tang, “Near-octave lithium niobate soliton microcomb,” Optica 7, 1275–1278 (2020).
  * [33] J. Zhang, Y. M. Sua, J.-Y. Chen, J. Ramanathan, C. Tang, Z. Li, Y. Hu, and Y.-P. Huang, “Carbon-dioxide absorption spectroscopy with solar photon counting and integrated lithium niobate micro-ring resonator,” Applied Physics Letters 118, 171103 (2021).
  * [34] A. Boes, B. Corcoran, L. Chang, J. Bowers, and A. Mitchell, “Status and potential of lithium niobate on insulator (lnoi) for photonic integrated circuits,” Laser & Photonics Reviews 12, 1700256 (2018).
  * [35] Q. Zhang, X.-H. Bao, C.-Y. Lu, X.-Q. Zhou, T. Yang, T. Rudolph, and J.-W. Pan, “Demonstration of a scheme for the generation of “event-ready” entangled photon pairs from a single-photon source,” Physical Review A 77, 062316 (2008).

## Acknowledgements

The research was supported in part by National Science Foundation (Award
#1641094 & #1842680 & #1806523) and National Aeronautics and Space
Administration (Grant Number 80NSSC19K1618). Device fabrication was performed
in Nanofabrication Facility at Advanced Science Research Center (ASRC), City
University of New York (CUNY).

## Author contributions statement

J. C. and Y. H. conceived the experiments. J. C. and Z. L. fabricated the
device and conducted the experiments. Z. M. and C. T. involved in the
fabrication. H. F. and Y. S. involved in the quantum measurement. J. C.
performed the numerical simulations. Y. H. supervised the project. All authors
contributed to write and review the manuscript.

## Competing interests

The authors declare no competing interests.

## Additional information

Correspondence and requests for materials should be addressed to J.C and Y.H.

Supplemental Materials: Photon conversion and interaction on chip

###  .1 Coupled-mode theory

The dynamic of sum-frequency generation (SFG) inside the $\chi^{(2)}$ cavity,
neglecting Rayleigh backscattering and assuming the single-mode condition, is
governed by the following coupled mode equations:

$\displaystyle\frac{da_{p}}{dt}$
$\displaystyle=(i\delta_{p}-\frac{\kappa_{p,t}}{2})a_{p}+ig^{*}a^{*}_{s}a_{f}+i\sqrt{\kappa_{p,c}}F_{p},$
(S1) $\displaystyle\frac{da_{s}}{dt}$
$\displaystyle=(i\delta_{s}-\frac{\kappa_{s,t}}{2})a_{s}+ig^{*}a^{*}_{p}a_{f}+i\sqrt{\kappa_{s,c}}F_{s},$
(S2) $\displaystyle\frac{da_{f}}{dt}$
$\displaystyle=(i(\delta_{p}+\delta_{s}+\Delta\omega)-\frac{\kappa_{f,t}}{2})a_{f}+iga_{p}a_{s},$
(S3)

where $\kappa_{j,o}=\omega_{j}/Q_{j,o}$ are the cavity dissipation rates,
$Q_{j,o}$ are the quality factors of cavity mode, $\omega_{j}$ is the angular
frequency, with $j=p,s,f$ for the pump, signal, sum-frequency modes,
respectively, and $o=0,c,t$ denoting intrinsic, coupling and total dissipation
rates with $\kappa_{t}=\kappa_{0}+\kappa_{c}$,
$\delta_{j}=\omega_{j}-\omega_{j,0}$ is the laser-cavity detuning and
$\omega_{j,0}$ is the cavity resonance, $F_{j}=\sqrt{N_{j}}$ with $N_{j}$ is
the photon number. Because of the energy conservation in parametric conversion
process, $\omega_{f}=\omega_{p}+\omega_{s}$ permits
$\delta_{p}+\omega_{p,0}+\delta_{s}+\omega_{s,0}=\delta_{f}+\omega_{f,0}$ thus
$\delta_{f}=\delta_{p}+\delta_{s}+\Delta\omega$ where
$\Delta\omega=\omega_{p,0}+\omega_{s,0}-\omega_{f,0}\approx 0$, when it is a
triply-resonant cavity. The photon-photon coupling coefficient $g$:

$\displaystyle
g=\sqrt{\frac{\hbar\omega_{p}\omega_{s}\omega_{f}}{2\epsilon_{0}\epsilon_{p}\epsilon_{s}\epsilon_{f}}}\frac{\frac{2}{\pi}d_{eff}\xi}{\sqrt{A_{eff}2\pi
R}}\times\delta(m_{f}-m_{p}-m_{s}-M),$ (S4) $\displaystyle\xi=\frac{\iint
E^{*}_{f}E_{p}E_{s}dxdy}{(\iint|E_{p}|^{2}E_{p}dxdy\iint|E_{s}|^{2}E_{s}dxdy\iint|E_{f}|^{2}E_{f}dxdy)^{1/3}},$
(S5)

where $\epsilon_{0}$ is the vacuum permittivity, $\epsilon_{j}=n^{2}_{j}$ are
the relative permittivity with $n_{j}$ giving the effective refractive
indices, $d_{eff}$ is the effective nonlinear susceptibility, $\xi$ is the
mode overlapping factor and $V_{eff}$ is the effective mode volume, $m_{j}$
are the azimuthal order of the cavity modes, M is the azimuthal poling grating
number. For quasi-phase matched case, $m_{f}-m_{p}-m_{s}-M=0$.

At steady state, the output filed of SF is defined as
$b_{f}=i\sqrt{\kappa_{f,c}}a_{f}$ and the quantum efficiency of photon
conversion is given by $\eta_{\mathrm{QE}}=N_{f}/N_{s}=\lvert
b_{f}\rvert^{2}/N_{s}$. The intracavity pump and signal photon number are
given by $\lvert a_{p}\rvert^{2}$ and $\lvert a_{s}\rvert^{2}$, respectively.
The above quantum efficiency gives the probability of a signal photon at the
cavity input being converted to its sum-frequency and appear at the cavity
output. It has accounted for the signal coupling loss and the SF extraction
loss. It is thus a direct measure of the device figure of merit that dictates
its performance in applications.

Another measure, of less practical relevance but an intrinsic property, is the
internal Rabi rotation angle $\theta=2gQ_{p,i}/\omega_{p}$. The system
$\hat{H}_{\textrm{eff}}=\hbar\mathnormal{g}(\hat{a}_{p}\hat{a}_{s}\hat{a}_{f}^{\dagger}+\hat{a}_{p}^{\dagger}\hat{a}_{s}^{\dagger}\hat{a}_{f})$
can be interpreted as the pump induced Rabi oscillation between signal and SF
photons. $\theta$ then gives how much Rabi rotation can a pump photon induce
during it is lost in the cavity.

We implement time split-step method to numerically solve the coupled-mode
equations. The parameters used in the simulation for Fig.3 and Fig.4 is listed
in Table.S1:

Parameters | Description | Value | Unit
---|---|---|---
$\kappa_{p,t}$ | total dissipation rate of pump mode | $1467.9$ | MHz ($\times 2\pi$)
$\kappa_{p,0}$ | intrinsic dissipation rate of pump mode | $270.8$ | MHz ($\times 2\pi$)
$\kappa_{p,c}$ | coupling dissipation rate of pump mode | $1197.0$ | MHz ($\times 2\pi$)
$\kappa_{s,t}$ | total dissipation rate of signal mode | $1510.3$ | MHz ($\times 2\pi$)
$\kappa_{s,0}$ | intrinsic dissipation rate of signal mode | $304.4$ | MHz ($\times 2\pi$)
$\kappa_{s,c}$ | coupling dissipation rate of signal mode | $1205.9$ | MHz ($\times 2\pi$)
$\kappa_{f,t}$ | total dissipation rate of sum-frequency mode | $1512.2$ | MHz ($\times 2\pi$)
$\kappa_{f,0}$ | intrinsic dissipation rate of sum-frequency mode | $318.7$ | MHz ($\times 2\pi$)
$\kappa_{f,c}$ | coupling dissipation rate of sum-frequency mode | $1193.5$ | MHz ($\times 2\pi$)
$d^{a}_{eff}$ | effective nonlinear susceptibility | 16.4 | pm/V
$d^{b}_{eff}$ | effective nonlinear susceptibility | 18.2 | pm/V
$A_{eff}$ | effective mode area | 1.0 |  $\mu$m2
$R$ | radius of the microring | 80 |  $\mu$m
$\xi$ | mode overlapping factor | 90% | NA
Table S1: Simulation parameters used in coupled-mode equations models for sum-
frequency generation. a: used in Fig.3, b: used in Fig.4. The fitted $g^{a}$
and $g^{b}$ are 8.2 MHz and 9.1 MHz, respectively.

###  .2 Experimental setup

We use the setup shown in Fig.S1 to characterize the device and perform the
photon conversion and interaction experiment. The whole chip is placed on a
thermoelectric cooler and the temperature is set at about 45 ∘C. For linear
characterization, as shown in Fig.S1(a), we use two polarized tunable
continuous-wave (CW) lasers (Santec 550 and Newport TLB-6712) and tapered
fibers (OZ OPTICS) to independently characterize the fiber-chip-fiber
coupling, whose losses are measured to be (8 $\pm$ 0.15) dB around 1556 nm and
(9.5 $\pm$ 0.2) dB around 778 nm, respectively. For its nonlinear optical
properties, we sweep the infrared laser across the whole C-band while
optimizing the chip’s temperature to achieve strong second-harmonic generation
(SHG).

Once identifying the quasi-phase matching resonances, we switch to the setup
in Fig.S1(b) for sum-frequency generation. Additional tunable CW laser
(Coherent, MTP-1000) is used to serve as the signal. We will further optimize
the system around its optimum SHG condition to maximize the SFG.

  * •

For classical measurement, optical spectrum analyzer (OSA, Yokogawa AQ6370D)
is used to collect the data.

  * •

For quantum noise measurement, a silicon-based single-photon detector
(Excelitas, efficiency 50$\%$, dark count 250 Hz) and a free-space filtering
system are introduced. It consist of two fiber collimators (insertion loss,
IL$\sim$ 2 dB), one short-pass filter ( IL$\sim$0.5 dB, extinction ratio, ER
$\sim$50 dB) and one band-pass filter (10nm, 780 nm, IL$\sim$0.5 dB, ER
$\sim$50 dB) rejecting pump and signal in C-band while passing through visible
light, and three narrow band-pass filters (Alluxa, 3 nm, IL$\sim$1 dB,
ER$>$120 dB) rejecting residue second-harmonic light of 780 nm while passing
through the target light of 778 nm.

  * •

For interaction between single-photon coherent states, superconducting
nanowire single-photon detector (ID Quantique, ID281, efficiency 85$\%$, dark
count 100 Hz) are used to monitor the input photon flux. Due to limited
photon-count saturation rate ($\sim$20 MHz) of SNSPDs, each monitor channel
has about 50 dB attenuation. Meanwhile, the free-space filtering system will
be removed to reduce the insertion loss (IL$\sim$4 dB). The SF photons will be
directly measured by silicon-based single-photon detector via a lensed fiber.

Figure S1: Setups for classical (a) and quantum (b) experiments. The blue and
red lines denote the telecome light path and visible path, respectively. FPC:
Fiber Polarization Controller; TEC: thermoelectric cooler; WDM: wavelength-
division multiplexing module; PD: photodetector; DUT: device under test;
VOA:variable optical attenuator; ATT: optical attenuator; OSA: optical
spectrum analyzer; FC: fiber collimator; SP: short-pass filter; BP: band-pass
filter; NBP: narrow band-pass filter; SNSPD: superconducting nanowire single-
photon detector; Si-SPD: silicon single-photon detector.


# Effect Size Estimation in Linear Mixed Models

Jürgen Groß  Institute for Mathematics and Applied Informatics, University of
Hildesheim, Germany<EMAIL_ADDRESS>and  Annette Möller
Faculty of Business Administration and Economics, Bielefeld University,
Germany<EMAIL_ADDRESS>
###### Abstract.

In this note, we reconsider Cohen’s effect size measure $f^{2}$ under linear
mixed models and demonstrate its application by employing an artificially
generated data set. It is shown how $f^{2}$ can be computed with the
statistical software environment R using lme4 without the need for
specification and computation of a coefficient of determination.

###### Key words and phrases:

Hypothesis testing, effect size, Cohen’s f2, linear regression, linear mixed
model, multivariate normal distribution

###### 2010 Mathematics Subject Classification:

62J05, 62J20, 62F03

Support of the second author by the Helmholtz Association’s pilot project
”Uncertainty Quantification” is gratefully acknowledged.

##  1\. Introduction

In studies with a large number of observations, statistical testing procedures
are prone to detect even minor departures from a null hypothesis yielding very
small p-values. However, since significance does not automatically imply
relevance, measures for the size of the effect associated with a possible
rejection of the null hypothesis are often recommended as a useful tool, see
e.g. Wilkinson (1999).

A well known effect size measure in a regression context when a quantitative
variable $Y$ depends on independent regressors (quantitative and/or
qualitative) is Cohen’s $f^{2}$, see Cohen (1988, Chapt. 9). Under
multivariate normality this measure is strongly related to an $F$ test of a
linear hypothesis that a subset $B$ of independent variables does not
substantially contribute to the explanation of $Y$, given a set $A$ of
independent regressors already included in the model. One may argue that even
if the regression coefficients associated with variables $B$ significantly
differ from zero, their contribution may be assessed as relevant when some
meaningful size of their effect is measured.

In this light studies may additionally want to report $f^{2}$ values for each
variable in a regression model given the others, see for example Tables 2 to 5
in Taylor et al. (2020). With regard to the procedure of calculating $f^{2}$,
the authors refer to Selya et al. (2012) who introduced a practical method in
relation with SAS® software. This approach even carries over to linear mixed
models, i.e. the case that some independent variables are associated with
random effects rather than with fixed effects. Hence, the original approach by
Cohen is generalized to a certain extent, involving the additional estimation
of a variance-covariance matrix.

In the following we reconsider and discuss the generalization of $f^{2}$ to
linear mixed models. We also point to an alternative computational procedure
which turns out to be especially useful in context with the statistical
software environment R (R Core Team, 2022) and well known package lme4, see
Bates et al. (2015), for fitting linear mixed models. Our explanations are
illustrated on the basis of an artificially generated data set.

##  2\. Linear Mixed Model

Consider a linear mixed model (LMM) described by

(1) $\bm{y}=\bm{X}\bm{\beta}+\bm{Z}\bm{u}+\bm{e}\;,$

where $\bm{y}$ is an $n\times 1$ observable random vector. It is assumed that
the $n\times p$ model matrix $\bm{X}$ of full column rank $p$ can be
partitioned as

(2) $\bm{X}=(\bm{1}_{n}:\bm{X}_{1}:\bm{X}_{2})\;,$

where $\bm{1}_{n}$ denotes the $n\times 1$ vector of ones, while the $n\times
p_{1}$ and $n\times p_{2}$ matrices $\bm{X}_{1}$ and $\bm{X}_{2}$ contain the
values of $p_{1}+p_{2}=p-1$ regressors. The $p\times 1$ vector $\bm{\beta}$
comprises $p$ unknown parameters $\beta_{0},\beta_{1},\ldots,\beta_{p-1}$
addressed as fixed effects. The $n\times q$ matrix $\bm{Z}$ contains the
values of independent variables associated with an $q\times 1$ vector $\bm{u}$
of unobservable random effects. It is assumed that $\bm{u}$ has expectation
$\bm{0}_{q}$ (the $q\times 1$ vector of zeroes) and variance covariance matrix
$\mathop{\operator@font Cov}\nolimits(\bm{u})=\sigma^{2}\bm{D}$ with unknown
parameter $\sigma^{2}>0$ and $q\times q$ matrix $\bm{D}$, which may depend on
further unknown parameters. For the $n\times 1$ vector $\bm{e}$ of
unobservable random errors it is assumed that ${\operator@font
E}(\bm{e})=\bm{0}_{n}$ and $\mathop{\operator@font
Cov}\nolimits(\bm{e})=\sigma^{2}\bm{T}$, where the $n\times n$ positive
definite matrix $\bm{T}$ may also depend on unknown parameters. Moreover, the
assumption $\mathop{\operator@font Cov}\nolimits(\bm{u},\bm{e})=\bm{0}_{q,n}$
(the $q\times n$ matrix of zeroes) implies

(3) $\mathop{\operator@font
Cov}\nolimits(\bm{y})=\sigma^{2}\bm{V},\quad\bm{V}=\bm{Z}\bm{D}\bm{Z}^{T}+\bm{T}\;.$

Then the above model may also be represented by the triplet
$\\{\bm{y},\bm{X}\bm{\beta},\sigma^{2}\bm{V}\\}$ and can be considered as a
special case of the general Gauss-Markov model, see e.g. Groß (2004). As a
matter of fact, formulas useful under model 1) carry over from a classical
regression model
$\bm{Q}^{-1}\bm{y}=\bm{Q}^{-1}\bm{X}\bm{\beta}+\bm{\varepsilon}$ with
${\operator@font E}(\bm{\varepsilon})=\bm{0}_{n}$ and $\mathop{\operator@font
Cov}\nolimits(\bm{\varepsilon})=\sigma^{2}\bm{I}_{n}$, see Christensen (2020,
Sect 2.7). Here $\bm{Q}$ denotes some nonsingular matrix satisfying
$\bm{Q}\bm{Q}^{T}=\bm{V}$. Let us assume for the moment that $\bm{V}$ is
completely known. Then

(4)
$\widehat{\bm{\beta}}=(\bm{X}^{T}\bm{V}^{-1}\bm{X})^{-1}\bm{X}^{T}\bm{V}^{-1}\bm{y}$

is the best linear unbiased estimator for $\bm{\beta}$, and

(5)
$\widehat{\sigma}^{2}=\frac{1}{\nu}(\bm{y}-\bm{X}\widehat{\bm{\beta}})^{T}\bm{V}^{-1}(\bm{y}-\bm{X}\widehat{\bm{\beta}}),\quad\nu=n-p\;,$

is the usual unbiased estimator for $\sigma^{2}$. Consider the linear
hypothesis $H_{0}:\bm{R}\bm{\beta}=\bm{r}$ versus
$H_{1}:\bm{R}\bm{\beta}\not=\bm{r}$ for a given $r\times p$ matrix $\bm{R}$ of
full row rank and a given $r\times 1$ vector $\bf{r}$. The corresponding $F$
statistic in model (1) is

(6)
$F=\frac{(\bm{R}\widehat{\bm{\beta}}-\bm{r})^{T}(\bm{R}\bm{B}\bm{R}^{T})^{-1}(\bm{R}\widehat{\bm{\beta}}-\bm{r})}{r\widehat{\sigma}^{2}}=\frac{(\bm{R}\widehat{\bm{\beta}}-\bm{r})^{T}(\bm{R}\bm{B}\bm{R}^{T})^{-1}(\bm{R}\widehat{\bm{\beta}}-\bm{r})}{(\bm{y}-\bm{X}\widehat{\bm{\beta}})^{T}\bm{V}^{-1}(\bm{y}-\bm{X}\widehat{\bm{\beta}})}\cdot\frac{\nu}{r}\;,$

where

(7) $\mathop{\operator@font
Cov}\nolimits(\widehat{\bm{\beta}})=\sigma^{2}\bm{B},\quad\bm{B}=(\bm{X}^{T}\bm{V}^{-1}\bm{X})^{-1}\;.$

Under multivariate normality the statistic $F$ follows a $F_{r,f}$
distribution provided $H_{0}$ holds true.

###  2.1. Effect Size

Suppose that we are interested in the effect of the independent variables
represented by the model matrix $\bm{X}_{1}$, given the variables represented
by $\bm{X}_{2}$. The corresponding linear hypothesis reads
$\bm{R}_{1}\bm{\beta}=\bm{0}_{p_{1}}$ with
$\bm{R}_{1}=(\bm{0}_{p_{1}}:\bm{I}_{p_{1}}:\bm{0}_{p_{1},p_{2}})$. Hence, by
reasoning similar to Cohen (1988), an appropriate measure for the size of the
effect based on the above $F$ statistic is provided by

(8)
$f^{2}=\frac{(\bm{R}_{1}\widehat{\bm{\beta}})^{T}(\bm{R}_{1}\bm{B}\bm{R}_{1}^{T})^{-1}(\bm{R}_{1}\widehat{\bm{\beta}})}{(\bm{y}-\bm{X}\widehat{\bm{\beta}})^{T}\bm{V}^{-1}(\bm{y}-\bm{X}\widehat{\bm{\beta}})}\;,$

thereby removing the factor $\nu/r$ from the $F$ statistic. This generalizes
Cohen’s $f^{2}$ in the sense that if $\bm{D}=\bm{0}_{q,q}$, then the
unobservable random vector $\bm{u}$ equals $\bm{0}_{q}$ with probability one,
the LMM reduces to the usual linear model of fixed effects only, and $f^{2}$
becomes identical to the measure provided by formula (9.2.1) in Cohen (1988).

We note that it is also possible to compute $f^{2}$ as

(9) $f^{2}=\frac{R_{A,B}^{2}-R_{A}^{2}}{1-R_{A,B}^{2}}$

for appropriately defined coefficients of determination $R_{A,B}^{2}$ derived
under the full model and $R_{A}^{2}$ derived under a reduced LMM assuming that
variables represented by $\bm{X}_{1}$ are not present at all. Such a formula
is the basis for the widely applied computational procedure suggested by Selya
et al. (2012).

###  2.2. Operational Effect Size

Formula (8) for $f^{2}$ is only operational when $\bm{V}$ is completely known,
a condition not met in practical applications. According to Harville (1977) a
simple LMM is the ordinary mixed and random effects ANOVA model, also referred
to as traditional variance components model, see Christensen (2019, Chapt. 5).
Under this model, the variance-covariance matrix of $\bm{y}$ depends on a
total of $m$ variance components. The $n\times q$ matrix $\bm{Z}$ is
partitioned as $\bm{Z}=(\bm{Z}_{1}:\cdots:\bm{Z}_{m-1})$, where each $n\times
q_{i}$ matrix $\bm{Z}_{i}$ is the design matrix corresponding to a qualitative
variable with a certain number of levels. In such a case one may assume

(10)
$\bm{D}=\text{diag}\left[(\sigma_{1}^{2}/\sigma^{2})\bm{I}_{q_{1}},\ldots,(\sigma_{m-1}^{2}/\sigma^{2})\bm{I}_{q_{m-1}}\right]$

where $\sigma^{2}>0$ and $\sigma_{1}^{2},\ldots\sigma_{m-1}^{2}\geq 0$ are $m$
unknown variance components. Then

(11)
$\bm{V}=\bm{I}_{n}+\sum_{i=1}^{m-1}(\sigma_{i}^{2}/\sigma^{2})\bm{Z}_{i}\bm{Z}_{i}^{T}\;.$

If $\sigma_{i}^{2}=0$ for some $i$, then the corresponding $q_{i}\times 1$
random effect vector $\bm{u}_{i}$ equals $\bm{0}_{q_{i}}$ with probability
one. Such a model, and also more general ones, may be fitted in R with the
package lme4, see Bates et al. (2015). From the fitting procedure its is
possible to obtain an estimate for the variance-covariance matrix

(12) $\widehat{\mathop{\operator@font
Cov}\nolimits}({\widehat{\bm{\beta}}})=\widehat{\sigma}^{2}\widehat{\bm{B}}$

of the estimated fixed effects parameter vector $\bm{\beta}$. Then a
corresponding estimate for $f^{2}$ is given as

(13)
$f^{2}=\frac{1}{\nu}(\bm{R}_{1}\widehat{\bm{\beta}})^{T}(\bm{R}_{1}\widehat{\mathop{\operator@font
Cov}\nolimits}({\widehat{\bm{\beta}}})\bm{R}_{1}^{T})^{-1}(\bm{R}_{1}\widehat{\bm{\beta}})\;.$

where the factor $1/\nu$ is required to compensate for the usage of
$\widehat{\sigma}^{2}$ in the estimated variance-covariance matrix (12). The
application of this formula is illustrated in the following section. The merit
of formula (13) lies in the fact that it can be applied whenever an estimate
(12) is available. But then, of course, the actual outcome also depends on the
estimation procedure, implying that different estimation methods for (12) may
also lead to (slightly) different actual values of (13). This, however, is not
the topic of our discussion. Also, when applying (13) there is no need for
computing coefficients of determination. Nonetheless it is possible to define
an operational version of (9) which corresponds to (13). This is demonstrated
in Sect. 3.4, where the $R^{2}$ measure discussed in Edwards et al. (2008) is
used with an operational version of the $F$ statistic obtained in the same
light as (13) by employing the very same variance-covariance estimate (12).
Again, employing alternative computational methods or alternative measures
$R^{2}$ in linear mixed models, see also Nakagawa and Schielzeth (2013);
Nakagawa et al. (2017), may result in different actual values of $f^{2}$.

Figure 1. Distribution of response variable $Y$ over all $n=1000$ observations
(histogram) and within two groups of sizes $n_{1}=687$ and $n_{2}=313$
indicated by $0$ and $1$ (boxplots)

##  3\. Example

In the following we discuss the performance of measure $f^{2}$ for an
artificially generated data set of $n=1000$ observations intended to further
illustrate some computational aspects. For the sake of simplicity our model
consists of two independent variables $X_{1}$ (categorical/binary) and $X_{2}$
(quantitative) associated with fixed effects and one variable $Z$
(categorical) associated with random effects. However, the same principles
apply when $X_{1}$, $X_{2}$, and $Z$ are extended to possible sets of
variables containing more than one element. Our setting corresponds to the
above mentioned variance components model with $p_{1}=p_{2}=1$ and $m=2$.

###  3.1. Variable of Interest

Figure 1 shows a discernible location difference with respect to the
distribution of the response variable $Y$ in the two groups indicated by the
binary variable $X_{1}$. The Welch two-sample $t$ statistic for the null
hypothesis of no difference in group means reads $|t|=6.0751$ implying a
highly significant result. A corresponding effect size measure is Cohen’s $d$
which may be computed from R package effectsize, see Ben-Shachar et al.
(2020), as $|d|=0.4122$. From Cohen, values $|d|=0.2$, $|d|=0.5$ and $|d|=0.8$
indicate a small, medium and large effect, respectively.

Figure 2. Scatter plot of $n=1000$ pairs $(X_{2},Y)$ with group markings
according to $X_{1}$

###  3.2. Additional Fixed Effects

From Figure 2 one may conclude, however, that the difference in the two groups
may to some extent be explained by the variable $X_{2}$, since there is a
tendency for larger values of $X_{2}$ to come along with larger values of $Y$
and observations from group 1 of variable $X_{1}$. Therefore one might be
interested in the size of the effect of $X_{1}$ when $X_{2}$ is held constant.
This can be achieved by considering a regression model with $X_{1}$ and
$X_{2}$ as independent variables and deriving the measure $f^{2}$ as explained
in (Cohen, 1988, Sect. 9). From package effectsize one gets $f^{2}=0.0017767$.
Here, values $f^{2}=0.02$, $f^{2}=0.15$ and $f^{2}=0.35$ are supposed to
indicate a small, medium and large effect, respectively.

Recently, Groß and Möller (2023) considered a generalized version $d_{\ast}$
of $d$ as an effect size measure for a binary variable $X_{1}$ given further
variables. It may be computed from $f^{2}$ as

(14) $d_{\ast}=\sqrt{f^{2}(n-2-w)\gamma}\;,$

where in our analysis $w=1$ is the number of additional independent variables
incorporated in the model and $\sigma^{2}\gamma$ is the variance of the
regression coefficient for $X_{1}$. For our data $\gamma=0.0065821$, yielding
$d_{\ast}=0.108$ and thus confirming a less than small effect.

Figure 3. Scatterplot of $(\\#\\{X_{1}=1\\},\overline{Y})$ for the 15 groups
of $Z$, where $\\#\\{X_{1}=1\\}$ denotes the number of observations with
$X_{1}=1$

###  3.3. Additional Fixed and Random Effects

As a next step one may take the categorical variable $Z$, admitting 15 groups
in our data set, into account. From Figure 3 it is seen that there is a
tendency for $Z$ groups with larger means of the response variable $Y$ to
contain less observations marked as 1 (referring to variable $X_{1}$) than $Z$
groups with smaller means of $Y$. However, since observations from group 1
were earlier seen to reveal on average larger $Y$ values than observations
from group 0, holding $Z$ constant is expected to contribute in favor of an
effect of $X_{1}$ again.

As noted before, the variable $Z$ is meant to be associated with a random
effects vector $\bm{u}$. The corresponding design matrix $\bm{Z}$ has 15
columns where each row contains 0’s and a single 1, indicating the membership
of the observations to the respective $Z$ variable group. There are two
variance components, the overall $\sigma^{2}>0$ and the random effects
variance denoted by $\sigma_{u}^{2}\geq 0$ here. The model may also be
reparameterized via an unknown $k\geq 0$ by setting
$\sigma_{u}^{2}=k\sigma^{2}$. This gives variance-covariance matrix

(15) $\mathop{\operator@font
Cov}\nolimits(y)=\sigma^{2}\bm{V},\quad\bm{V}=k\bm{Z}\bm{Z}^{T}+\bm{I}_{m}\;,$

and the LMM reduces to the usual fixed effects regression model in case $k=0$.
For $\bm{V}$ from (15) one may compute the effect size $f^{2}$ for $X_{1}$ as
a function of $k$ from formula (8) when all variables $X_{1}$, $X_{2}$ and $Z$
are included in the LMM formulation. Figure 4 shows $f^{2}$ for different
choices of $k$. As expected, the effect size is larger when both, $X_{2}$ and
$Z$ are incorporated into the model compared to the case when only $X_{2}$ is
employed, corresponding to the choice $k=0$.

Figure 4. Values of $f^{2}$ from a LMM fit depending on $k$

The operational version of $f^{2}$ from (13) can easily be computed from
fitting the model by the function lmer from package lme4 as fit <\- lmer(Y ~ 1
+ X1 + X2 + (1|Z)). The estimated variance components are
$\widehat{\sigma}^{2}=393.4455$ and $\sigma_{u}^{2}=180.4234$ giving
$\widehat{k}=\sigma_{u}^{2}/\widehat{\sigma}^{2}=0.4586$ as an estimation for
$k$, see the dashed vertical line in Figure 4. Then the vector
$\widehat{\beta}$ is obtained from fixef(fit) and
$\widehat{\mathop{\operator@font Cov}\nolimits}({\widehat{\bm{\beta}}})$ is
obtained from vcov(fit). By using $\bm{R}_{1}=(0,\,1,\,0)$ and $\nu=n-p=997$,
formula 13 yields $f^{2}=0.0946626$ indicating a small but not medium effect
size of $X_{1}$ when $X_{2}$ and $Z$ are held constant.

###  3.4. Coefficient of Determination

Finally, we confirm that $f^{2}$ may also be obtained from formula (9),
although there is no actual need for this when formula (13) can be used
instead. For this, we define

(16) $R_{AB}^{2}=\frac{(r/\nu)F}{1+(r/\nu)F},\quad r=p-1,\,\nu=n-p\;,$

which is the proposed $R^{2}$ for linear mixed models by Edwards et al. (2008,
Eq. (19)). Here, $F$ is the $F$ statistic from (6) for testing the hypothesis
$H_{0}:\bm{R}\bm{\beta}=\bm{0}_{p-1}$ with
$\bm{R}=(\bm{0}_{p-1}:\bm{I}_{p-1})$. The relationship between $F$ and
$R_{AB}^{2}$ from (16) may be established similar to Example 4.8 from Seber
and Lee (2003). For our data $\bm{R}=(\bm{0}_{2}:\bm{I}_{2})$, $r=2$, and
$\nu=n-p=997$. The measure $R_{A}^{2}$ is defined in the same way, but for a
reduced model with all variables present except for $X_{1}$. For this reduced
model we have $\bm{R}=(0,\,1)$, $\bm{r}=(0)$, $r=1$, $\nu=n-p_{2}-1=998$. This
results in

(17)
$f^{2}=\frac{R_{A,B}^{2}-R_{A}^{2}}{1-R_{A,B}^{2}}=\frac{0.1539263-0.07418754}{1-0.1539263}=0.09424569\;,$

which is nearly the same as the value computed above. The displayed values
were obtained from operational versions of $R_{A,B}^{2}$ and $R_{A}^{2}$
computed in the same light as formula (13) by employing the estimated variance
covariance matrix of the fixed effects resulting from applying vcov() to the
two models in question.

## References

  * Bates et al. (2015) D. Bates, M. Mächler, B. Bolker, and S. Walker.  Fitting linear mixed-effects models using lme4.  _Journal of Statistical Software_ , 67:1–48, 2015.  URL https://doi.org/10.18637/jss.v067.i01.
  * Ben-Shachar et al. (2020) M. S. Ben-Shachar, D. Lüdecke, and D. Makowski.  effectsize: Estimation of effect size indices and standardized parameters.  _Journal of Open Source Software_ , 5:2815, 2020.  URL https://doi.org/10.21105/joss.02815.
  * Christensen (2019) R. Christensen.  _Advanced Linear Modeling: Statistical Learning and Dependent Data_.  Springer Nature, 2019.
  * Christensen (2020) R. Christensen.  _Plane Answers to Complex Questions. Fifth Edition_.  Springer, 2020.
  * Cohen (1988) J. Cohen.  _Statistical Power Analysis for the Behavioral Sciences. Second Edition_.  Lawrence Erlbaum Associates, 1988.
  * Edwards et al. (2008) L. J. Edwards, K. E. Muller, R. D. Wolfinger, B. F. Qaqish, and O. Schabenberger.  An R2 statistic for fixed effects in the linear mixed model.  _Statistics in Medicine_ , 27:6137–6157, 2008.
  * Groß (2004) J. Groß.  The general Gauss-Markov model with possibly singular dispersion matrix.  _Statistical Papers_ , 45:311–336, 2004.
  * Groß and Möller (2023) J. Groß and A. Möller.  A note on Cohen’s d from a partitioned linear regression model.  _Journal of Statistical Theory and Practice_ , 17:22, 2023\.  URL https://doi.org/10.1007/s42519-023-00323-w.
  * Harville (1977) D. A. Harville.  Maximum likelihood approaches to variance component estimation and to related problems.  _Journal of the American Statistical Association_ , 72:320–338, 1977.
  * Nakagawa and Schielzeth (2013) S. Nakagawa and H. Schielzeth.  A general and simple method for obtaining R2 from generalized linear mixed-effects models.  _Methods in Ecology and Evolution_ , 4:133–142, 2013.  URL https://doi.org/10.1111/j.2041-210x.2012.00261.x.
  * Nakagawa et al. (2017) S. Nakagawa, P. C. Johnson, and H. Schielzeth.  The coefficient of determination R 2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded.  _Journal of the Royal Society Interface_ , 14:20170213, 2017\.  URL https://doi.org/10.1098/rsif.2017.0213.
  * R Core Team (2022) R Core Team.  _R: A Language and Environment for Statistical Computing_.  R Foundation for Statistical Computing, Vienna, Austria, 2022.  URL https://www.R-project.org/.
  * Seber and Lee (2003) G. A. Seber and A. J. Lee.  _Linear Regression Analysis_.  John Wiley & Sons, 2003.
  * Selya et al. (2012) A. S. Selya, J. S. Rose, L. C. Dierker, D. Hedeker, and R. J. Mermelstein.  A practical guide to calculating Cohen’s f2, a measure of local effect size, from PROC MIXED.  _Frontiers in Psychology_ , 3:111, 2012.  URL https://doi.org/10.3389/fpsyg.2012.00111.
  * Taylor et al. (2020) S. Taylor, C. A. Landry, M. M. Paluszek, and G. J. Asmundson.  Reactions to COVID-19: Differential predictors of distress, avoidance, and disregard for social distancing.  _Journal of Affective Disorders_ , 277:94–98, 2020\.  URL https://doi.org/10.1016/j.jad.2020.08.002.
  * Wilkinson (1999) L. Wilkinson.  Statistical methods in psychology journals: Guidelines and explanations.  _American Psychologist_ , 54:594–604, 1999.