# Sensitivity potential to a light flavor-changing scalar boson with DUNE and
NA64$\mu$

B. Radics<EMAIL_ADDRESS>Department of Physics and Astronomy, York
University, Toronto, ON, Canada  L. Molina-Bueno  Instituto de Física
Corpuscular, Universidad de Valencia and CSIC, Carrer del Catedtrátic José
Beltrán Martinez, 2, 46980 Paterna, Valencia, Spain  L. Fields  Department of
Physics and Astronomy, University of Notre Dame, IN, 46556, USA  H. Sieber  P.
Crivelli  ETH Zürich, Institute for Particle Physics and Astrophysics, CH-8093
Zürich, Switzerland

###### Abstract

In this work, we report on the sensitivity potential of complementary muon-on-
target experiments to new physics using a scalar boson benchmark model
associated with charged lepton flavor violation. The NA64$\mu$ experiment at
CERN uses a 160-GeV energy muon beam with an active target to search for
excess events with missing energy and momentum as a probe of new physics. At
the same time, the proton beam at Fermilab, which is used to produce the
neutrino beam for the Deep Underground Neutrino Experiment (DUNE) will also
produce a high-intensity muon beam dumped in an absorber. Combined with the
liquid Argon Near Detector, the system could be used to search for similar
scalar boson particles with a lower energy but higher intensity beam. We find
that both NA64$\mu$ and DUNE could cover new, unexplored parts of the
parameter space of the same benchmark model, providing a complementary way to
search for new physics.

##  I Introduction

Observations in cosmology and astrophysics imply the existence of a Dark
Sector potentially containing new particles that could weakly couple to
Standard Model (SM) particles Lanfranchi  _et al._ (2021). Neutrino
oscillations coupled with non-zero neutrino masses provide an experimental
evidence of lepton-flavor violation. Furthermore, the existing discrepancy
between the measured Abi  _et al._ (2021) and expected Aoyama  _et al._
(2020); Borsanyi  _et al._ (2021) value of the muon anomalous magnetic moment
provides a strong motivation for new physics searches with muons Chen  _et
al._ (2017). Inspired by these developments, a certain class of new theories
proposes the search for charged lepton flavor violation (CLFV), which is
heavily suppressed in the Standard Model (SM). In the coming decades, a new
generation of experiments will conduct experimental searches for CLFV Meucci
(2022); Chiappini  _et al._ (2021); Bartoszek  _et al._ (2014); Hesketh  _et
al._ (2022); Abramishvili  _et al._ (2020); Altmannshofer  _et al._ (2019). In
parallel, next-generation long-baseline neutrino oscillation experiments Abud
_et al._ (2022); Abe  _et al._ (2015) will study neutrino oscillations and at
the same time produce an intense muon beam-dump with its beam line.

In this work we study the sensitivity potential of muon-on-target experiments
to new physics using a CLFV benchmark model. The physics scenario was
introduced in a recent work Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023) and
uses a light, scalar boson associated with $\mu-\tau$ conversion. While the
previous work derived constraints from data in existing beam-dump experiments
(LSNDAuerbach  _et al._ (2001), NuTeV Adams  _et al._ (2001), CHARM Dorenbosch
_et al._ (1986)) and for the future SHiP experiment Ale (2016), here we focus
on two further experiments: the coming neutrino experiment, DUNE Abud  _et
al._ (2022) deploying a high-intensity beam from the Long-Baseline Neutrino
Facility (LBNF) Abi  _et al._ (2020), and a fixed-target experiment, NA64$\mu$
Gninenko (2019); Gninenko  _et al._ (2019a), which searches for new physics
with the high-energy muon beam from the CERN Super Proton Synchrotron (SPS)
accelerator. Hence, we study new physics searches using two complementary
muon-beam setups. Even though we focus on one particular benchmark model
scenario, there are also other possibilities to probe hidden sectors with muon
beams and using similar techniques Gninenko  _et al._ (2019b, 2018, c);
Gninenko and Krasnikov (2022); Sieber  _et al._ (2023).

##  II Flavour-changing scalar with long lifetime

The model proposed by Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023) considers
a new complex scalar field, $\phi$, with a mass window
$[m_{\tau}-m_{\mu},m_{\tau}+m_{\mu}]$ that couples to $(\mu,\tau)$. With such
a mass range, long lifetimes can be achieved with propagation distances on the
order of tens of kilometers. The effective Lagrangian interaction terms
describe the coupling of the new scalar field with leptons,

$\mathcal{L_{I}}=\phi\bar{\mu}(g_{V}+g_{A}\gamma^{5})l+\phi^{*}\bar{l}(g_{V}^{*}-g_{A}^{*}\gamma^{5})\mu$
(1)

with vector and axial vector coupling $g_{V}$, $g_{A}$. This model produces a
benchmark parameter region explaining the muon $g-2$ anomaly with a typical
value of $|g_{V}|\simeq 3\times 10^{-3}$, also used in this work. There are
multiple production modes to produce $\phi$ at beam dump experiments: direct
electroweak process, heavy meson decay and high-energy muons on hitting a
fixed target. In this work, we focus on the third case (the so-called
$\mu$-on-target scenario). In this mode, muons pass through dense material and
could produce the $\phi$ boson via the exchange of a virtual photon with the
nuclei of the target,
$\mu(p)N(P_{i})\rightarrow\tau(p^{\prime})\phi(k)N(P_{f})$. Once created, the
bosons produce a missing energy signature or propagate long distances and may
be detected in a detector downstream from the target. When the incoming beam
energy is much higher than the particle mass, the double-differential cross-
section of the $2\rightarrow 3$ production process can be estimated using the
equivalent photon approximation Bjorken  _et al._ (2009),

$\displaystyle\frac{d^{2}\sigma(p+P_{i}\rightarrow
p^{\prime}+k+P_{f})}{dE_{\phi}d\cos\theta_{\phi}}$ $\displaystyle=$ (2)
$\displaystyle\frac{\alpha\chi}{\pi}\frac{E_{\mu}x\beta_{\phi}}{1-x}$
$\displaystyle\frac{d\sigma(p+q\rightarrow p^{\prime}+k)}{d(p\cdot
k)}\biggr{\rvert}_{t=t_{\mathrm{min}}}$

where $E_{\mu}$ is the initial muon energy, $E_{\phi}$ is the energy and
$\theta_{\phi}$ the scattering angle of $\phi$ with respect to the initial
muon in the lab frame, $x=E_{\phi}/E_{\mu}$, $q=P_{i}-P_{f}$, $t=-q^{2}$ is
the momentum transfer, $\alpha=1/137$ is the fine structure constant,
$\beta_{\phi}$ is the relativistic factor for $\phi$, and $\chi$ is the
effective photon flux, defined as

$\chi=\int_{t_{\mathrm{min}}}^{t_{\mathrm{max}}}dt\frac{t-t_{\mathrm{min}}}{t^{2}}F^{2}(t)$
(3)

where $F(t)=Z^{2}/(1+t/d)^{2}$ is the form factor with $d=0.164$
GeV${}^{2}A^{-2/3}$, and the limits are given in the appendix of Y. Ema, Z.
Liu, K-F Lyu and M. Pospelov (2023). We calculate the analytical expression
for $\chi$ using Mathematica Inc.  and obtain

$\chi=Z^{2}\left[\frac{t_{\mathrm{min}}}{t}+\frac{d+t_{\mathrm{min}}}{d+t}+\frac{(d+2t_{\mathrm{min}})}{d}\frac{\ln{t}}{\ln{(d+t)}}\right]_{t_{\mathrm{min}}}^{t_{\mathrm{max}}}.$
(4)

In Eq. 2, $\sigma(p+q\rightarrow p^{\prime}+k)$ is the cross section for the
$2\rightarrow 2$ scattering process,
$\mu(p)\gamma(q)\rightarrow\tau(p^{\prime})\phi(k)$,

$\frac{d\sigma(p+q\rightarrow p^{\prime}+k)}{d(p\cdot
k)}=\frac{|\mathcal{\bar{A}}_{2\rightarrow 2}|^{2}}{8\pi s^{2}}$ (5)

where $|\mathcal{\bar{A}}|^{2}$ is the amplitude squared, which we calculate
using the FeynCalc tools Shtabovenko  _et al._ (2020) for Mathematica. We
obtain the following expression for the $(\mu,\tau)$ case,

$\displaystyle|\mathcal{\bar{A}}_{2\rightarrow
2}|^{2}=-\frac{e^{2}m_{\mu}m_{\tau}(g_{A}g_{A}^{*}-g_{V}g_{V}^{*})}{(m_{\mu}^{2}-s)^{2}(m_{\tau}^{2}-u)^{2}}\times$
(6)
$\displaystyle\times[m_{\mu}^{4}(m_{\phi}^{2}+u)+2m_{\mu}^{3}(m_{\tau}^{3}-m_{\tau}u)$
$\displaystyle+m_{\mu}^{2}\left(m_{\tau}^{4}-2m_{\phi}^{2}s-2m_{\tau}^{2}u+u(u-2s)\right)$
$\displaystyle+2m_{\mu}m_{\tau}s(u-m_{\tau}^{2})+s\left(m_{\phi}^{2}s+m_{\tau}^{4}-2m_{\tau}^{2}u+u(s+u)\right)]$

where $e=\sqrt{4\pi\alpha}$, $m_{\phi}$, $m_{\tau}$, $m_{\mu}$ are the masses
of the boson $\phi$, $\tau$ and $\mu$, respectively, and $s$, $u$ are the
Mandelstam variables, which can be evaluated in the laboratory frame,

$\displaystyle s=(p+q)^{2}\simeq m_{\mu}^{2}-\frac{u-m_{\tau}^{2}}{1-x}$ (7)
$\displaystyle u=(p-k)^{2}\simeq-
E_{\mu}x\theta_{\phi}^{2}-\frac{1-x}{x}m_{\phi}^{2}+(1-x)m_{\mu}^{2}.$

To calculate the lifetime of the $\phi$ boson, $\tau_{\phi}$, we use Equations
3.2-3.5 in Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023) adapted to the
$(\mu,\tau)$ case. We calculate the cross-section and decay-width using the
GNU Scientific Library Galassi  _et al._ (2018). The production cross-section
for the $\phi$ boson as a function of the incoming lepton energy is shown in
Fig. 1. Assuming $m_{\phi}\simeq m_{\tau}$, the threshold for the production
is given by
$E_{\mu}>[(2m_{\tau}+m_{N})^{2}-m_{\mu}^{2}-m_{N}^{2}]/2m_{N}\simeq 3.8$ GeV
for Pb, above which the cross-section steeply rises. In the following, we use
the corresponding muon beam flux of each experiment (NA64$\mu$ and DUNE) to
evaluate the expressions for the $\phi$ production via the variables defined
in Eq. 7.

$0$$50$$100$$150$$10^{-7}$$10^{-6}$$10^{-5}$$10^{-4}$$10^{-3}$$E_{\mu}$
[GeV]Cross section [$10^{-36}$ cm2] Figure 1: Production cross-section of the
$\phi$ boson, with mass $m_{\phi}=m_{\tau}$ and coupling constant
$|g_{V}|=3\times 10^{-3}$, as a function of the incoming muon energy.

##  III $\mu$-on-target experiments

We estimate the projected sensitivity of an experiment by finding the pair of
parameter values $(g_{V},m_{\phi})$, where $N_{\phi}$ appropriate signal
events are produced either directly in the target (NA64$\mu$) or in the
detector (DUNE), as explained later, after a given exposure. In this work we
exploit the $\phi$ boson production process using different and complementary
$\mu$-on-target experiments: NA64$\mu$ with the active dump technique compared
to proton beam-dump experiments such as the DUNE neutrino experiments using
LBNF proton beam. Although the underlying production mechanism is the same,
the two techniques differ in the flux of the muon beam, $\Phi_{\mu}(E)$, the
target thickness and materials.

In a general case, $\phi$ bosons are generated by the $\mu$-on-target process,
and, after production, a fraction of them decay inside a detector volume and
can be detected. The number of such signal events is

$N_{\phi}=\int
dE_{\phi}\Phi_{\phi}(E_{\phi})\times\frac{l_{\mathrm{det}}}{\gamma\beta
c\tau_{\phi}},$ (8)

where $\gamma$ is the relativistic Lorentz-factor, and $\Phi_{\phi}(E_{\phi})$
is the flux of the $\phi$ boson at the detector, estimated as

$\displaystyle\Phi_{\phi}(E_{\phi})=\int dE\Phi_{\mu}(E)\times$ (9)
$\displaystyle\int_{E_{\mathrm{min}}}^{E}dE_{l}\frac{n_{A}}{-dE/dl}\int_{0}^{\theta_{\mathrm{det}}}d\theta_{\phi}\sin\theta_{\phi}\frac{d^{2}\sigma(E_{l},E_{\phi})}{dE_{\phi}d\cos\theta_{\phi}}.$

Here, $\Phi_{\mu}(E)$ is the flux of the muon beam as a function of energy,
$n_{A}$ is the number of target atoms per volume, $E_{l}$ is the muon energy
after traveling a length $l$ in the target and losing energy according to the
stopping power $-dE/dl$, $E_{\mathrm{min}}$ is the energy of the muon at the
end of the target, and $\theta_{\mathrm{det}}$ is the angular acceptance.

In the next subsections we separately describe the two experimental scenarios
and the assumptions in the derived sensitivity limits.

###  III.1 NA64$\mu$ scenario

NA64$\mu$ is a fixed-target experiment at CERN looking for new particles of
Dark Matter and portal interactions produced in electromagnetic showers and
coupled to muons. The experiment uses the secondary 160 GeV muons from the
interactions of 400 GeV protons from the CERN SPS with a target. A set of beam
scintillators and veto counters, low material-budget trackers and dipole
magnets allow to precisely constrain the momentum of the incoming 160-GeV
muons impinging on an active target. The main detector, where $\phi$ may be
produced, consists of an electromagnetic calorimeter with 40 $X_{0}$ radiation
length. Downstream, the detector is further equipped with veto counters and a
$\sim$30-interaction length hadronic calorimeter. New particles could be
produced by the muon beam scattering in the target and decay later to visible
SM particles that could be seen by their signatures in a downstream detector.
The current work is based on the detection of missing energy and momentum
carried away by the produced hypothetical, long-lived $\phi$ boson, leaving a
scattered muon as experimental signature (the momentum of the scattered muon
ranging between $10-80$ GeV/c). The sensitivity in the search for the $\phi$
boson is higher with respect to the beam-dump approach due to the lower power
in the coupling strength without a decay vertex. Thus, in NA64$\mu$ only the
number of events at the production target needs to be estimated. Therefore,
the number of events is given by $N_{\phi}=\int
dE_{\phi}\Phi_{\phi}(E_{\phi})$. Furthermore, the production target thickness
is small and the muon energy loss can be neglected. As a result we use the
following expression to estimate the $\phi$ boson flux,

$\displaystyle\Phi_{\phi}(E_{\phi})=l_{\mathrm{target}}n_{A}\int
dE\Phi_{\mu}(E)\times$ (10)
$\displaystyle\int_{0}^{\theta_{\mathrm{det}}}d\theta_{\phi}\sin\theta_{\phi}\frac{d^{2}\sigma(E,E_{\phi})}{dE_{\phi}d\cos\theta_{\phi}},$

where $l_{\mathrm{target}}$ is the thickness of the target.

There has been an extensive study of simulating and evaluating the production
of Dark Matter particles with a muon beam at NA64 Bondi  _et al._ (2021);
Kirpichnikov  _et al._ (2021); Sieber  _et al._ (2023). We use the same method
in this work to estimate the sensitivity reach of the experiment. We assume a
muon beam with a mean of 160 GeV energy and a width of 4.3 GeV hitting a lead
target with a total data of $\sim 3\times 10^{13}$ muons-on-target (MOT).

###  III.2 DUNE/LBNF scenario

The Deep Underground Neutrino Experiment is a next-generation, wide-energy
beam long-baseline neutrino experiment at Fermilab. It will use an intense
(anti)neutrino beam that passes through a Near Detector at Fermilab and a Far
Detector 1300 km away in South Dakota. The neutrino beam line of DUNE is a
result of an exhaustive design and optimization work Abi  _et al._ (2020). The
beam is produced by a 60-120 GeV proton beam hitting a graphite target after
which the produced pions and kaons decay to leptons and neutrinos in a $\sim
220$-m-long decay pipe. At the end of pipe a dedicated $\sim 30$-m-long
stainless-steel structure acts as a beam dump to stop all muons $300$ m
upstream from the Near Detector. We use the simulation of the neutrino beam
production to trace particles along the beam axis. The muon flux used in the
calculation is estimated from a dedicated tracking plane, which is located at
the end of the decay pipe in the simulation. An example of the obtained muon
energy spectrum is illustrated in Fig. 2. The peak of the spectrum is at
$E_{\mu}\simeq 2.5$ GeV, however, the long high-energy tail is responsible for
the majority of $\phi$ bosons production due to the rapid increase of the
cross-section with the incoming lepton energy (see Fig. 1). At the lower
energy region the production rate is much smaller. From the neutrino flux
simulation we estimate an integrated muon flux of $\Phi_{\mu}\simeq 5\times
10^{19}$ muons for $1.1\times 10^{21}$ Proton-On-Target (POT), corresponding
to one year of data taking.

In DUNE, a signal could be detected in the Near Detector from the decay of the
$\phi$ boson that was produced by the muons hitting the stainless-steel dump .
We consider the decay channel
$\phi\rightarrow\mu^{+}\mu^{-}\nu_{\mu}\nu_{\tau}$ with a branching ratio of
$17\%$ Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023), motivated by the dimuon
results from NuTeV Adams  _et al._ (2001). Possible backgrounds leading to a
dimuon signature include deep inelastic scattering (DIS) and resonance
production of mesons in charged-current (CC) muon-neutrino ($\nu_{\mu}$)
interactions with a target nucleus. The mesons could decay in semi-leptonic
mode, producing an extra muon. In order to analyze such potential background
processes we performed simulations with the GENIE Andreopoulos  _et al._
(2010) Monte Carlo (MC) event generator that provides a comprehensive neutrino
interaction modeling in the $E_{\nu}\sim 100$ MeV $-$ few 100 GeV neutrino
energy region, including quasi-elastic, resonance and DIS processes. Unlike
the $\phi$ boson decay, events with DIS or resonant meson production are
accompanied with additional activity in the final state. Similarly to previous
findings for NuTeV, after discriminating for low-multiplicity events with two
muons, we found that these backgrounds could be completely suppressed in a MC
simulation of 400 million events of neutrino interactions on an Argon target
(corresponding to $\sim 5$ years of operation of DUNE at the nominal
intensity). However, further studies are planned with a full detector
simulation to get a detailed understanding of the possible bounds on the
background rejection.

$0$$20$$40$$60$$80$$10^{0}$$10^{1}$$10^{2}$$10^{3}$$10^{4}$$10^{5}$$E_{\mu}$
[GeV]$\Phi_{\mu}(E$) [a.u.] Figure 2: Energy spectrum of muons at the end of
the decay pipe from the full DUNE neutrino beam line simulation Abi  _et al._
(2020), see explanation in the text.

##  IV Results

We illustrate the sensitivity potential for the benchmark CLFV scenario with
the complementary muon beams at NA64$\mu$ and DUNE in Fig 3. For both
experiments, the double-differential cross-section in Eq. 9 or 10 is evaluated
given the energy spectra of each experiment, $\Phi_{\mu}(E)$, and the
kinematical limits on the final-state $\phi$-boson fractional energy, $x$,
which is constrainted by the masses of the boson $\phi$ and the $\tau$.

In the case of the NA64$\mu$ experiment a total integrated muon flux of $\sim
3\times 10^{13}$ MOT is achievable Gninenko (2019). The time needed to
accumulate the assumed total MOT is estimated to be $\sim 100$ days. This
conservative estimation is based on the CERN SPS delivering on average 3500
spills per day and $2\times 10^{8}$ muons per spill. Benefiting from the
unique combination of a 160-GeV muon beam with missing-energy and momentum
search, NA64$\mu$ would be able to perform a competitive search for such a
CLFV signal. Furthermore, the sensitivity reach critically depends on the
length of the active target. We find that a 1-m-long target would already be
able to explore a large part of the parameter space, $g_{V}\geq 6\times
10^{-3}$. However, the experiment is highly modular and a possible
optimization of the setup could enhance its potential further: a feasible
option is to increase the target length to 5 meters which would allow to
completely cover the $(g_{\mu}-2)$ preferred region and to probe a variety of
other new physics scenarios involving muons. We also find that already with a
$10^{12}$ MOT and a $\sim 3$ m long target the $g_{V}\geq\times 10^{-2}$
parameter region can be covered. A detailed Monte Carlo simulation of an
optimized setup will follow as a next step.

For the DUNE experiment the $\phi$-boson production is driven by the high-end
tail of the muon flux. Compared to NA64$\mu$, the lower muon energies at DUNE
and thus lower production cross-section are compensated by the more intense
muon flux. Assuming 20 years of operation at nominal intensity, DUNE would be
able to explore a significant part of the parameter space, reaching into the
$g_{V}\simeq 10^{-2}$ region and potentially improving the constraints from
NuTeV. However, an optimization of the neutrino beam line could further
enhance the contribution of high-energy muons in the flux and subsequently
approach the $g_{V}\leq 10^{-2}$ benchmark region. This scenario is partially
motivated by a recent work exploring an alternative beam-dump operation mode
to probe new physics with DUNE Brdar  _et al._ (2023).

For comparison we also show the projected sensitivity for the same $\mu$-on-
target mode calculated by Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023) for
CHARM, NuTeV and SHiP. Here we do not show the constraints or projected
sensitivity limits derived for the direct electroweak and heavy meson decay
process, which was included in the previous work Y. Ema, Z. Liu, K-F Lyu and
M. Pospelov (2023) since we only focus on the $\mu$-on-target scenario. In the
case of SHiP, the assumed total data corresponds to $2\times 10^{20}$ protons-
on-target (POT). The worse sensitivity in DUNE compared to SHiP stems from the
lower muon energies in the flux Ahd (2020). It is also noted that there are
differences in the beam intensity between NA64$\mu$ and SHiP. We assume a
$10^{12}$ POT per spill intensity at the CERN SPS in the case of the muon beam
line used by NA64$\mu$, while for SHiP the proton beam intensity is usually
expected to be an order of magnitude higher.

We note that a similar setup of NA64$\mu$ is capable of searching for other
new scalar particle candidates using the same muon beam Sieber  _et al._
(2023). In addition, the proposed Muon Missing Momentum ($M^{3}$) experiment
at Fermilab Kahn  _et al._ (2018) also plans to probe new physics with a
dedicated muon beam. Finally, a number of experiments also have the potential
to search for hidden-sector scalar particles, such as SHADOWS Alviggi  _et
al._ (2022), HIKE Cortina Gil  _et al._ (2022), and ATLAS Galon  _et al._
(2020).

$1.7$$1.75$$1.8$$1.85$$10^{-3}$$10^{-2}$$10^{-1}$SHiPDUNENA64$\mu$(1
m)NA64$\mu$(5 m)$\mu$ g-2 $\pm 2\sigma$ CHARMNuTeV$m_{\phi}$ [GeV]gV Figure 3:
Constraints from CHARM, NuTeV Adams  _et al._ (2001); Dorenbosch  _et al._
(1986), and projected sensitivity curves for SHiP Y. Ema, Z. Liu, K-F Lyu and
M. Pospelov (2023); Ale (2016) (dash-dotted, magenta line), DUNE (solid, blue
line) and NA64$\mu$ (dashed, brown and black lines), in the $\mu$-on-target
production mode (except for CHARM). In the case of NA64$\mu$ the optimized
setup with 1-m-long (brown dashed line) and 5-m-long (black dashed line)
targets are also shown separately. The light blue region shows the benchmark
parameter range explaining the muon $g_{\mu}-2$ anomaly.

##  V Conclusions

In summary, we present the sensitivity potential of two $\mu$-on-target
experiments: NA64$\mu$ and DUNE, as complementary modes of searching for new
physics with muon beams. We find that both NA64$\mu$ and DUNE have the
potential to cover a significant portion of the benchmark model parameter
space, ($m_{\phi},g_{V}$). NA64$\mu$ with an optimized setup could probe the
coupling parameter down to $g_{V}\simeq 3\times 10^{-3}$, completely covering
the muon $g_{\mu}-2$ preferred region and thus providing a similar projected
reach as SHiP. DUNE will also be able to cover unexplored parts of the
parameter space, potentially improving on the obtained constraints from NuTeV.
An optimization of the neutrino beam line, increasing the contribution from
the high-energy tail, could allow to further enhance the sensitivity of DUNE.

Although we use a given CLFV model as a benchmark in this work, we note that
similar techniques can be used to study the sensitivity potential of
experiments with muon beams to other physics scenarios. Beyond the $g_{\mu}-2$
discrepancy and neutrino flavor oscillations, searches for Dark Sector
particles are also motivated by the matter-antimatter asymmetry, the known
Dark Matter abundance from astrophysical and cosmological observations, or by
theoretical motivations strongly suggesting the existence of additional gauge
groups weakly coupling to SM fields Lanfranchi  _et al._ (2021); Antel  _et
al._ (2023).

###### Acknowledgements.

We gratefully acknowledge conversations with D. Harris. The work of LMB is
supported by SNSF Grant No. 186158 (Switzerland), RyC-030551-I, and
PID2021-123955NA-100 funded by MCIN/AEI/ 10.13039/501100011033/FEDER, UE
(Spain).

## References

  * Lanfranchi  _et al._ (2021) G. Lanfranchi, M. Pospelov, and P. Schuster, Annual Review of Nuclear and Particle Science 71, 279 (2021).
  * Abi  _et al._ (2021) B. Abi _et al._ (Muon $g-2$ Collaboration), Phys. Rev. Lett. 126, 141801 (2021).
  * Aoyama  _et al._ (2020) T. Aoyama _et al._ , Physics Reports 887, 1 (2020).
  * Borsanyi  _et al._ (2021) S. Borsanyi _et al._ , Nature 593, 51 (2021).
  * Chen  _et al._ (2017) C.-Y. Chen, M. Pospelov, and Y.-M. Zhong, Phys. Rev. D 95, 115005 (2017).
  * Meucci (2022) M. Meucci, PoS NuFact2021, 120 (2022).
  * Chiappini  _et al._ (2021) M. Chiappini _et al._ (MEG II Collaboration), Universe 7 (2021).
  * Bartoszek  _et al._ (2014) L. Bartoszek _et al._ (Mu2e),  (2014), 10.2172/1172555, arXiv:1501.05241 [physics.ins-det] .
  * Hesketh  _et al._ (2022) G. Hesketh, S. Hughes, A.-K. Perrevoort, and N. Rompotis (Mu3e), in  _Snowmass 2021_ (2022) arXiv:2204.00001 [hep-ex] .
  * Abramishvili  _et al._ (2020) R. Abramishvili _et al._ (COMET), PTEP 2020, 033C01 (2020), arXiv:1812.09018 [physics.ins-det] .
  * Altmannshofer  _et al._ (2019) W. Altmannshofer _et al._ (Belle-II), PTEP 2019, 123C01 (2019), [Erratum: PTEP 2020, 029201 (2020)], arXiv:1808.10567 [hep-ex] .
  * Abud  _et al._ (2022) A. A. Abud _et al._ (DUNE Collaboration), Phys. Rev. D 105, 072006 (2022).
  * Abe  _et al._ (2015) K. Abe _et al._ , Progress of Theoretical and Experimental Physics 2015, 053C02 (2015).
  * Y. Ema, Z. Liu, K-F Lyu and M. Pospelov (2023) Y. Ema, Z. Liu, K-F Lyu and M. Pospelov, JHEP 135 (2023).
  * Auerbach  _et al._ (2001) L. B. Auerbach _et al._ (LSND Collaboration), Phys. Rev. D 63, 112001 (2001).
  * Adams  _et al._ (2001) T. Adams _et al._ , Phys. Rev. Lett. 87, 041801 (2001).
  * Dorenbosch  _et al._ (1986) J. Dorenbosch _et al._ , Physics Letters B 166, 473 (1986).
  * Ale (2016) Reports on Progress in Physics 79, 124201 (2016).
  * Abi  _et al._ (2020) B. Abi _et al._ , “Deep Underground Neutrino Experiment (DUNE), Far Detector Technical Design Report, Volume II: DUNE Physics,” (2020), arXiv:2002.03005 [hep-ex] .
  * Gninenko (2019) S. Gninenko (NA64),  _Proposal for an experiment to search for dark sector particles weakly coupled to muon at the SPS_, Tech. Rep. (CERN, Geneva, 2019).
  * Gninenko  _et al._ (2019a) S. N. Gninenko _et al._ , Physics Letters B 796, 117 (2019a).
  * Gninenko  _et al._ (2019b) S. N. Gninenko, D. V. Kirpichnikov, and N. V. Krasnikov, Phys. Rev. D 100, 035003 (2019b), arXiv:1810.06856 [hep-ph] .
  * Gninenko  _et al._ (2018) S. Gninenko, S. Kovalenko, S. Kuleshov, V. E. Lyubovitskij, and A. S. Zhevlakov, Phys. Rev. D 98, 015007 (2018), arXiv:1804.05550 [hep-ph] .
  * Gninenko  _et al._ (2019c) S. N. Gninenko, D. V. Kirpichnikov, M. M. Kirsanov, and N. V. Krasnikov, Phys. Lett. B 796, 117 (2019c), arXiv:1903.07899 [hep-ph] .
  * Gninenko and Krasnikov (2022) S. N. Gninenko and N. V. Krasnikov (NA64), Phys. Rev. D 106, 015003 (2022), arXiv:2202.04410 [hep-ph] .
  * Sieber  _et al._ (2023) H. Sieber, D. V. Kirpichnikov, I. V. Voronchikhin, P. Crivelli, S. N. Gninenko, M. M. Kirsanov, N. V. Krasnikov, L. Molina-Bueno, and S. K. Sekatskii, “Probing hidden sectors with a muon beam: implication of spin-0 dark matter mediators for muon $(g-2)$ anomaly and validity of the weiszäcker-williams approach,” (2023), arXiv:2305.09015 [hep-ph] .
  * Bjorken  _et al._ (2009) J. D. Bjorken, R. Essig, P. Schuster, and N. Toro, Phys. Rev. D 80, 075018 (2009).
  * (28) W. R. Inc., “Mathematica, Version 13.1,” Champaign, IL, 2022\.
  * Shtabovenko  _et al._ (2020) V. Shtabovenko, R. Mertig, and F. Orellana, Computer Physics Communications 256, 107478 (2020).
  * Galassi  _et al._ (2018) M. Galassi _et al._ , “GNU Scientific Library Reference Manual,”  (2018).
  * Bondi  _et al._ (2021) M. Bondi _et al._ , Computer Physics Communications 269, 108129 (2021).
  * Kirpichnikov  _et al._ (2021) D. V. Kirpichnikov, H. Sieber, L. Molina Bueno, P. Crivelli, and M. M. Kirsanov, Phys. Rev. D 104, 076012 (2021).
  * Andreopoulos  _et al._ (2010) C. Andreopoulos _et al._ , Nucl. Instrum. Meth. A 614, 87 (2010), arXiv:0905.2517 [hep-ph] .
  * Brdar  _et al._ (2023) V. Brdar, B. Dutta, W. Jang, D. Kim, I. M. Shoemaker, Z. Tabrizi, A. Thompson, and J. Yu, Phys. Rev. D 107, 055043 (2023).
  * Ahd (2020) Eur. Phys. J. C 80, 284 (2020).
  * Kahn  _et al._ (2018) Y. Kahn, G. Krnjaic, N. Tran, and A. Whitbeck, JHEP 09, 153 (2018), arXiv:1804.03144 [hep-ph] .
  * Alviggi  _et al._ (2022) M. Alviggi _et al._ ,  _SHADOWS Search for Hidden And Dark Objects With the SPS_ , CERN-SPSC-2022-030/SPSC-I-256 (2022).
  * Cortina Gil  _et al._ (2022) E. Cortina Gil _et al._ (HIKE), (2022), arXiv:2211.16586 [hep-ex] .
  * Galon  _et al._ (2020) I. Galon, E. Kajamovitz, D. Shih, Y. Soreq, and S. Tarem, Phys. Rev. D 101, 011701(R) (2020), arXiv:1906.09272 [hep-ph] .
  * Antel  _et al._ (2023) C. Antel _et al._ , arXiv e-prints , arXiv:2305.01715 (2023), arXiv:2305.01715 [hep-ph] .


# New 3-mode squeezing operator and squeezed vacuum state in 3-wave mixing
††thanks: Work was supported by the National Natural Science Foundation of
China under grants 10775097 and the Key Programs Foundation of Ministry of
Education of China (No. 210115).

Xue-xiang Xu1,2, Hong-yi Fan1, Li-yun Hu2,and Hong-chun Yuan1
1Department of Physics, Shanghai Jiao Tong University, Shanghai, 200240, China
2College of Physics & Communication Electronics, Jiangxi Normal University,
Nanchang 330022, China<EMAIL_ADDRESS>
###### Abstract

In a 3-wave mixing process occurring in some nonlinear optical medium when
$a_{1}^{\dagger}$mode interacts with both $a_{2}^{\dagger}$mode and
$a_{3}^{\dagger}$mode, we theoretically study the squeezing effect generated
by the operator
$S_{3}\equiv\exp[\mu(a_{1}a_{2}-a_{1}^{\dagger}a_{2}^{\dagger})+\nu(a_{1}a_{3}-a_{1}^{\dagger}a_{3}^{\dagger})]$.
The new 3-mode squeezed vacuum state in Fock space is derived, and the
uncertainty relation for it is demonstrated, It turns out that $S_{3}$may
exhibit enhanced squeezing. By virtue of the technique of integration within
an ordered product (IWOP) of operators, we also derive $S_{3}$’s normally
ordered expansion. The Wigner function of new 3-mode squeezed vacuum state is
calculated by using the Weyl ordering invariance under similar
transformations.

PACS 42.50.-p – Quantum optics

PACS 03.65.-w – Quantum mechanics

##  1 Introduction

Nowadays quantum entanglement is the focus of quantum information research and
attracts many interests due to its wide applications in quantum communication
[1, 2]. Entangled states have brought much attention and interests of
physicists [3, 4]. The usual two-mode squeezed state, generated from a
parametric amplifier [5], not only exhibits squeezing, but also quantum
entanglement between the idle-mode and the signal-mode in frequency domain.
Therefore, it is simultaneously a typical entangled state of continuous
variable. Theoretically, the two-mode squeezed state is constructed by acting
a two-mode squeezing operator
$S_{2}=\exp[\lambda(a_{1}a_{2}-a_{1}^{\dagger}a_{2}^{\dagger})]$ [6, 7] on the
two-mode vacuum state $\left|00\right\rangle$, i.e.
$S_{2}\left|00\right\rangle=$sech$\lambda\exp[-a_{1}^{\dagger}a_{2}^{\dagger}\tanh\lambda]\left|00\right\rangle$,
where $\lambda\ $is a squeezing parameter, and $a_{i}$($a_{j}^{\dagger}$) Bose
annihilation (creation) operator satisfying
$[a_{i},a_{j}^{\dagger}]=\delta_{ij}$. Using the relation between Bose
operators ($a_{i},a_{i}^{\dagger}$) and the coordinate and momentum operators

$Q_{i}=\frac{a_{i}+a_{i}^{\dagger}}{\sqrt{2}},\
P_{i}=\frac{a_{i}-a_{i}^{\dagger}}{\sqrt{2}\mathtt{i}},$ (1)

one can recast $S_{2}$ into the form

$S_{2}=\exp\left[\mathtt{i}\lambda\left(Q_{1}P_{2}+Q_{2}P_{1}\right)\right],$
(2)

noting

$\begin{array}[c]{c}\left[Q_{1}P_{2},Q_{2}P_{1}\right]=\mathtt{i}\left(Q_{2}P_{2}-Q_{1}P_{1}\right),\\\
\left[Q_{1}P_{2},\mathtt{i}\left(Q_{2}P_{2}-Q_{1}P_{1}\right)\right]=2Q_{1}P_{2},\\\
\left[Q_{2}P_{1},\mathtt{i}\left(Q_{2}P_{2}-Q_{1}P_{1}\right)\right]=-2Q_{2}P_{1},\end{array}$
(3)

thus there involves a $SU(1,1)$ algebraic stricture. In the state
$S_{2}\left|00\right\rangle$, the variances of the two-mode quadrature
operators of light field,

$\mathfrak{X}=\frac{Q_{1}+Q_{2}}{2},\text{
}\mathfrak{P}=\frac{P_{1}+P_{2}}{2},$ (4)

satisfying the commutation relation
$[\mathfrak{X},\mathfrak{P}]=\frac{\mathtt{i}}{2}$, exhibiting the standard
squeezing, i.e.,

$\left\langle
00\right|S_{2}^{\dagger}\mathfrak{X}^{2}S_{2}\left|00\right\rangle=\frac{1}{4}e^{-2\lambda},\left\langle
00\right|S_{2}^{\dagger}\mathfrak{P}^{2}S_{2}\left|00\right\rangle=\frac{1}{4}e^{2\lambda},$
(5)

which satisfy $(\Delta\mathfrak{X})(\Delta\mathfrak{P})=\frac{1}{4}$.

An interesting question naturally arises: if $a_{1}^{\dagger}$ mode in a
nonlinear optical medium, interacting with both $a_{2}^{\dagger}$ mode and
$a_{3}^{\dagger}$ mode (e.g., a three-wave mixing), and the corresponding
three-mode exponential operator is introduced as

$S_{3}\equiv\exp[\mu(a_{1}a_{2}-a_{1}^{\dagger}a_{2}^{\dagger})+\nu(a_{1}a_{3}-a_{1}^{\dagger}a_{3}^{\dagger})].$
(6)

Using Eq.(1) we can recast $S_{3}$ into the form

$S_{3}\equiv\exp\left[\mathtt{i}\mu\left(Q_{2}P_{1}+Q_{1}P_{2}\right)+\mathtt{i}\nu\left(Q_{3}P_{1}+Q_{1}P_{3}\right)\right],$
(7)

where $\mu,$ $\nu$ are two different interaction parameters, then what is its
squeezing effect for the 3-mode quadratures of light field?

To answer this question we must know what is the state
$S_{3}\left|000\right\rangle$ ($\left|000\right\rangle$ is the 3-mode vacuum
state) in Fock space, for this aim, we should know what is the normally
ordered expansion of $S_{3}$. But how to disentangle the exponential operator
$S_{3}?$ Because there is no simple $SU(1,1)$ algebraic structure among
$Q_{2}P_{1},Q_{1}P_{2},Q_{3}P_{1}$ and $Q_{1}P_{3},$ the disentangling seems
hard. Thus we turn to appeal to Dirac’s coordinate representation and the
technique of integration within an ordered product (IWOP) of operators [8, 9,
10, 11] to solve this problem. Our work is arranged as follows: firstly we
derive the explicit form of$\ S_{3}\left|000\right\rangle,$ then we
demonstrate that it really satisfies the Heisenberg uncertainty relation and
may exhibit squeezing enhancement. We also employ the technique of integration
within an ordered product (IWOP) of operators to derive the normally ordered
expansion of $S_{3}$. The Wigner function of $S_{3}\left|000\right\rangle$ is
calculated by using the Weyl ordering invariance under similar transformations
[12, 13, 14].

##  2 New 3-mode squeezed vacuum state

For the sake of convenience, we rewrite $S_{3}$ in Eq.(7) as the following
compact form,

$S_{3}=\exp[\mathtt{i}Q_{i}\Lambda_{ij}P_{j}],i,j=1,2,3,$ (8)

where the repeated indices imply the Einstein summation notation, and

$\Lambda=\left(\begin{array}[c]{ccc}0&\mu&\nu\\\ \mu&0&0\\\
\nu&0&0\end{array}\right),$ (9)

thus

$e^{\Lambda}=\allowbreak\left(\begin{array}[c]{ccc}\cosh r&\cos\theta\sinh
r&\sin\theta\sinh r\\\ \cos\theta\sinh r&\sin^{2}\theta+\cos^{2}\theta\cosh
r&\frac{\sin 2\theta}{2}\left(\cosh r-1\right)\\\ \sin\theta\sinh r&\frac{\sin
2\theta}{2}\left(\cosh r-1\right)&\sin^{2}\theta\cosh
r+\cos^{2}\theta\end{array}\right),$ (10)

its inverse is

$e^{-\Lambda}\allowbreak=\allowbreak\left(\begin{array}[c]{ccc}\cosh
r&-\cos\theta\sinh r&-\sin\theta\sinh r\\\ -\cos\theta\sinh
r&\sin^{2}\theta+\cos^{2}\theta\cosh r&\frac{\sin 2\theta\left(\cosh
r-1\right)}{2}\\\ -\sin\theta\sinh r&\frac{\sin 2\theta\left(\cosh
r-1\right)}{2}&\sin^{2}\theta\cosh r+\cos^{2}\theta\end{array}\right),$ (11)

where we have set

$r=\sqrt{\mu^{2}+\nu^{2}},\cos\theta=\frac{\mu}{r},\sin\theta=\frac{\nu}{r},$
(12)

noting that $\Lambda$ is a symmetric matrix. Using the Baker-Hausdorff
formula,

$\displaystyle e^{A}Be^{-A}$
$\displaystyle=B+\left[A,B\right]+\frac{1}{2!}\left[A,\left[A,B\right]\right]$
$\displaystyle+\frac{1}{3!}\left[A,\left[A,\left[A,B\right]\right]\right]+\cdots,$
(13)

we see that $S_{3}$ causes the following transformation

$S_{3}^{-1}Q_{k}S_{3}=(e^{-\Lambda})_{ki}Q_{i},\text{\
}S_{3}^{-1}P_{k}S_{3}=(e^{\Lambda})_{ki}P_{i}.$ (14)

It then follows that $S_{3}^{-1}a_{k}S_{3}=(e^{-\lambda\Lambda})_{ki}a_{i}$,
i.e.

$\displaystyle S_{3}^{-1}a_{1}S_{3}$ $\displaystyle=a_{1}\cosh
r-a_{2}^{{\dagger}}\cos\theta\sinh r-a_{3}^{{\dagger}}\sin\theta\sinh r,$
$\displaystyle S_{3}^{-1}a_{2}S_{3}$
$\displaystyle=-a_{1}^{{\dagger}}\cos\theta\sinh
r+a_{2}\left(\sin^{2}\theta+\cos^{2}\theta\cosh r\right)$
$\displaystyle+\frac{1}{2}a_{3}\left(\cosh r-1\right)\sin 2\theta,$ (15)
$\displaystyle S_{3}^{-1}a_{3}S_{3}$
$\displaystyle=-a_{1}^{{\dagger}}\sin\theta\sinh r+\frac{1}{2}a_{2}\left(\cosh
r-1\right)\sin 2\theta$ $\displaystyle+a_{3}\left(\sin^{2}\theta\cosh
r+\cos^{2}\theta\right).$

Noticing that $S_{3}^{\dagger}=S_{3}^{-1}$ and
$S_{3}^{\dagger}\left(\mu,\nu\right)=S_{3}\left(-\mu,-\nu\right)$, from
Eq.(15) we also have

$\displaystyle S_{3}a_{1}S_{3}^{-1}$ $\displaystyle=a_{1}\cosh
r+a_{2}^{{\dagger}}\cos\theta\sinh r+a_{3}^{{\dagger}}\sin\theta\sinh r,$
$\displaystyle S_{3}a_{2}S_{3}^{-1}$
$\displaystyle=a_{1}^{{\dagger}}\cos\theta\sinh
r+a_{2}\left(\sin^{2}\theta+\cos^{2}\theta\cosh r\right)$
$\displaystyle+\frac{1}{2}a_{3}\left(\cosh r-1\right)\sin 2\theta$ (16)
$\displaystyle S_{3}a_{3}S_{3}^{-1}$
$\displaystyle=a_{1}^{{\dagger}}\sin\theta\sinh r+\frac{1}{2}a_{2}\left(\cosh
r-1\right)\sin 2\theta$ $\displaystyle+a_{3}\left(\sin^{2}\theta\cosh
r+\cos^{2}\theta\right).$

For convenience to write, we set
$S_{3}\left|000\right\rangle=\left\|000\right\rangle$. In order to obtain the
explicit form of $\left\|000\right\rangle$, using Eq.(15) and
$a_{1}\left|000\right\rangle=0$, we operate $a_{1}$ on
$\left\|000\right\rangle$ and obtain

$\displaystyle a_{1}\left\|000\right\rangle$
$\displaystyle=S_{3}S_{3}^{-1}a_{1}S_{3}\left|000\right\rangle$
$\displaystyle=S_{3}(a_{1}\cosh r-a_{2}^{{\dagger}}\cos\theta\sinh
r-a_{3}^{{\dagger}}\sin\theta\sinh r)\left|000\right\rangle$
$\displaystyle=S_{3}(-a_{2}^{{\dagger}}\cos\theta\sinh
r-a_{3}^{{\dagger}}\sin\theta\sinh r)S_{3}^{-1}S_{3}\left|000\right\rangle$
$\displaystyle=-S_{3}(a_{2}^{{\dagger}}\cos\theta\sinh
r+a_{3}^{{\dagger}}\sin\theta\sinh r)S_{3}^{-1}\left\|000\right\rangle,$ (17)

then we continue to use Eq.(16) to derive

$\displaystyle a_{1}\left\|000\right\rangle$
$\displaystyle=-\\{[a_{1}\cos\theta\sinh
r+a_{2}^{\dagger}\left(\sin^{2}\theta+\cos^{2}\theta\cosh r\right)$
$\displaystyle+\frac{1}{2}a_{3}^{\dagger}\left(\cosh r-1\right)\sin
2\theta]\cos\theta\sinh r$ $\displaystyle+[a_{1}\sin\theta\sinh
r+\frac{1}{2}a_{2}^{\dagger}\left(\cosh r-1\right)\sin 2\theta$
$\displaystyle+a_{3}^{\dagger}\left(\sin^{2}\theta\cosh
r+\cos^{2}\theta\right)]\sin\theta\sinh r\\}\left\|000\right\rangle$
$\displaystyle=-(a_{1}\sinh^{2}r+a_{2}^{\dagger}\allowbreak\cos\theta\cosh
r\sinh r$ $\displaystyle+\frac{1}{2}a_{3}^{\dagger}\sin\theta\sinh
2r)\left\|000\right\rangle,$ (18)

so we reach the equation

$a_{1}\left\|000\right\rangle=-\tanh
r(a_{2}^{\dagger}\allowbreak\cos\theta+a_{3}^{\dagger}\sin\theta)\left\|000\right\rangle.$
(19)

Similarly, operating $a_{2}$ on $\left\|000\right\rangle$ and using Eqs.(15)
and (16) yields

$\displaystyle a_{2}\left\|000\right\rangle$
$\displaystyle=S_{3}S_{3}^{-1}a_{2}S_{3}\left|000\right\rangle=S_{3}(-a_{1}^{{\dagger}}\cos\theta\sinh
r)\left|000\right\rangle$
$\displaystyle=S_{3}(-a_{1}^{{\dagger}}\cos\theta\sinh
r)S_{3}^{-1}\left\|000\right\rangle$ $\displaystyle=-(a_{1}^{{\dagger}}\cosh
r+a_{2}\cos\theta\sinh r$ $\displaystyle+a_{3}\sin\theta\sinh
r)\cos\theta\sinh r\left\|000\right\rangle,$ (20)

which leads to

$\displaystyle[a_{2}\left(1+\cos^{2}\theta\sinh^{2}r\right)+\frac{1}{2}a_{3}\sin
2\theta\sinh^{2}r]\left\|000\right\rangle$
$\displaystyle=-\frac{1}{2}a_{1}^{{\dagger}}\cos\theta\sinh
2r\left\|000\right\rangle.$ (21)

On the other hand, operating $a_{3}$ on $\left\|000\right\rangle$ and using
Eqs.(15) and (16) yields

$\displaystyle a_{3}\left\|000\right\rangle$
$\displaystyle=S_{3}S_{3}^{-1}a_{3}S_{3}\left|000\right\rangle=S_{3}(-a_{1}^{{\dagger}}\sin\theta\sinh
r)\left|000\right\rangle$
$\displaystyle=S_{3}(-a_{1}^{{\dagger}}\sin\theta\sinh
r)S_{3}^{-1}\left\|000\right\rangle$ $\displaystyle=-(a_{1}^{{\dagger}}\cosh
r+a_{2}\cos\theta\sinh r$ $\displaystyle+a_{3}\sin\theta\sinh
r)\sin\theta\sinh r\left\|000\right\rangle,$ (22)

i.e.,

$\displaystyle[a_{3}\left(1+\sin^{2}\theta\sinh^{2}r\right)+\frac{1}{2}a_{2}\sin
2\theta\sinh^{2}r]\left\|000\right\rangle$
$\displaystyle=-\frac{1}{2}a_{1}^{{\dagger}}\sin\theta\sinh
2r\left\|000\right\rangle.$ (23)

Combining Eqs.(21) and (23) we have

$a_{2}\left\|000\right\rangle=-a_{1}^{{\dagger}}\tanh
r\cos\theta\left\|000\right\rangle,$ (24)

and

$a_{3}\left\|000\right\rangle=-a_{1}^{{\dagger}}\tanh
r\sin\theta\left\|000\right\rangle.$ (25)

From Eqs.(19),(24) and (25), we may predict that $\left\|000\right\rangle$ has
the following explicit form:

$\left\|000\right\rangle=N\exp[-(a_{2}^{{\dagger}}\cos\theta+a_{3}^{{\dagger}}\sin\theta)a_{1}^{{\dagger}}\tanh
r]\left|000\right\rangle,$ (26)

where $N$ is the normalization constant, which can be determined by
$\left\langle 000\right.\left\|000\right\rangle=1$, and we calculate
$N=\sec$h$r.$

##  3 Squeezing property and quantum fluctuation in $\left\|000\right\rangle$

Squeezing is an important phenomenon in quantum theory and has many
applications in various areas in quantum optics and quantum information [15].
In this section, we examine the quadrature squeezing effects of
$\left\|000\right\rangle$. The quadratures in the 3-mode case are defined as

$X_{1}=\frac{1}{\sqrt{6}}\sum_{i=1}^{3}Q_{i},\text{
}X_{2}=\frac{1}{\sqrt{6}}\sum_{i=1}^{3}P_{i},$ (27)

which satisfy the relation $[X_{1},X_{2}]=\frac{\mathtt{i}}{2}.$ Their
variances are $\left(\Delta X_{i}\right)^{2}=\left\langle
X_{i}^{2}\right\rangle-\left\langle X_{i}\right\rangle^{2}$, $i=1,2.$ Noting
the expectation values of $X_{1}$ and $X_{2}$ in the state
$\left\|000\right\rangle$ is $\left\langle X_{1}\right\rangle=\left\langle
X_{2}\right\rangle=0$. With the help of Eq.(15), we can calculate that the
corresponding variances in the state $\left\|000\right\rangle$: (noting
$\Lambda$ is symmetric)

$\displaystyle\left(\triangle X_{1}\right)^{2}$ $\displaystyle=\left\langle
000\right|S_{3}^{-1}X_{1}^{2}S_{3}\left|000\right\rangle$
$\displaystyle=\frac{1}{6}\sum_{i=1}^{3}\sum_{j=1}^{3}(e^{-\Lambda})_{ki}(e^{-\Lambda})_{jl}\left\langle
000\right|Q_{k}Q_{l}\left|000\right\rangle$
$\displaystyle=\frac{1}{12}\sum_{i=1}^{3}\sum_{j=1}^{3}(e^{-\Lambda})_{ki}(e^{-\Lambda})_{jl}\left\langle
000\right|a_{k}a_{l}^{\dagger}\left|000\right\rangle$
$\displaystyle=\frac{1}{12}\sum_{i=1}^{3}\sum_{j=1}^{3}(e^{-\Lambda})_{ki}(e^{-\Lambda})_{jl}\delta_{kl}$
$\displaystyle=\frac{1}{12}\underset{i,j}{\sum^{3}}(e^{-2\Lambda})_{ij},$ (28)

and

$\left(\triangle X_{2}\right)^{2}=\left\langle
000\right|S_{3}^{-1}X_{2}^{2}S_{3}\left|000\right\rangle=\frac{1}{12}\underset{i,j}{\sum^{3}}(e^{2\Lambda})_{ij}.$
(29)

The explicit form of the matrices $e^{2\Lambda}$ and $e^{-2\Lambda}$ can be
derived from Eq.(10) and (11), so we can obtain

$\displaystyle\left(\triangle X_{1}\right)^{2}$
$\displaystyle=\frac{1}{12}[(2\cosh 2r+1)+\sin 2\theta(\allowbreak\cosh 2r-1)$
$\displaystyle+2\left(\cos\theta+\sin\theta\right)\sinh 2r],$ (30)

and

$\displaystyle\left(\triangle X_{2}\right)^{2}$
$\displaystyle=\frac{1}{12}[(2\cosh 2r+1)+\sin 2\theta(\allowbreak\cosh 2r-1)$
$\displaystyle-2\left(\cos\theta+\sin\theta\right)\sinh 2r].$ (31)

We can successfully verify

$\displaystyle(\triangle X_{1})(\triangle X_{2})$
$\displaystyle=\frac{1}{12}\sqrt{\left(4\cosh
2r+4\right)+\left(1-2\sinh^{2}r\sin 2\theta\right)^{2}}$
$\displaystyle\geqslant\frac{1}{12}\sqrt{\left(4\cosh
2r+4\right)+\left(1-2\sinh^{2}r\right)^{2}}$
$\displaystyle=\frac{1}{12}\sqrt{\frac{1}{2}\cosh
4r+\frac{17}{2}}\geqslant\frac{1}{4},$ (32)

which confirms the uncertainty relation of quantum mechanics.

To see the trend of squeezing effects in the $X_{1}-$ or $X_{2}-$direction, we
plot $\left(\Delta X_{1}\right)^{2}$ and $\left(\Delta X_{2}\right)^{2}$ as
the function of parameter $\mu$ for different $\nu$ in Fig.1. When $\nu=0,$ it
exhibits the usual two mode squeezing effect depending on the varying $\mu$,
$\left(\Delta X_{1}\right)^{2}$ increases accompanying $\left(\Delta
X_{2}\right)^{2}$ decreases; when $\nu=0.5$, $\left(\Delta X_{1}\right)^{2}\
$increases more than the case of $\nu=0$, which exhibits enhanced squeezing in
certain domain of $\mu$. In Fig.2, we plot the uncertainty value $(\triangle
X_{1})(\triangle X_{2})$ as the function of $r$ for different $\theta$.

##  4 Normally ordered form of $S_{3}$

We calculate the normally ordered form (denoted by $\colon\colon$) of $S_{3}$
by inserting the completeness relation of coherent state

$S_{3}=\int\frac{d^{2}z_{1}d^{2}z_{2}d^{2}z_{3}}{\pi^{3}}S_{3}\left|z_{1}z_{2}z_{3}\right\rangle\left\langle
z_{1}z_{2}z_{3}\right|.$ (33)

where $\left|z_{1}z_{2}z_{3}\right\rangle$ is the three-mode coherent state
and
$\left|z_{i}\right\rangle=\exp[-\frac{\left|z_{i}\right|^{2}}{2}+z_{i}a_{i}^{{\dagger}}]\left|0_{i}\right\rangle$,
$i=1,2,3$.

Using the relations in Eqs.(15) and (16), we have the explicit relation of
$S_{3}\left|z_{1}z_{2}z_{3}\right\rangle\allowbreak$

$\displaystyle S_{3}\left|z_{1}z_{2}z_{3}\right\rangle$
$\displaystyle=\exp(-\sum_{i=1}^{3}\frac{\left|z_{i}\right|^{2}}{2})S_{3}\exp\left(z_{1}a_{1}^{\dagger}+z_{2}a_{2}^{\dagger}+z_{3}a_{3}^{\dagger}\right)S_{3}^{-1}S_{3}\left|000\right\rangle$
$\displaystyle=\frac{1}{\cosh
r}\exp(-\sum_{i=1}\frac{\left|z_{i}\right|^{2}}{2}+z_{1}z_{2}\cos\theta\tanh
r+z_{1}z_{3}\sin\theta\tanh r)$
$\displaystyle\times\exp\allowbreak\\{\frac{1}{\cosh
r}[a_{1}^{{\dagger}}z_{1}+\allowbreak(a_{2}^{{\dagger}}\left(\sin^{2}\theta\cosh
r+\cos^{2}\theta\right)-\frac{1}{2}a_{3}^{{\dagger}}\left(\cosh r-1\right)\sin
2\theta)z_{2}$
$\displaystyle+(a_{3}^{{\dagger}}\left(\sin^{2}\theta+\cos^{2}\theta\cosh
r\right)-\frac{1}{2}a_{2}^{{\dagger}}\left(\cosh r-1\right)\sin
2\theta)z_{3}]\\}$ $\displaystyle\times\exp[-a_{1}^{{\dagger}}\tanh
r(a_{2}^{{\dagger}}\cos\theta+a_{3}^{{\dagger}}\sin\theta)]\left|000\right\rangle.$
(34)

Substituting Eq.(34) into Eq.(33), noticing that
$\left|000\right\rangle\left\langle
000\right|=\colon\exp(-a_{1}^{{\dagger}}a_{1}-a_{2}^{{\dagger}}a_{2}-a_{3}^{{\dagger}}a_{3})\colon$,
and using the following formula

$\int\frac{d^{2}z}{\pi}\exp\left(\zeta\left|z\right|^{2}+\xi z+\eta
z^{\ast}\right)=-\frac{1}{\zeta}e^{-\frac{\xi\eta}{\zeta}},\text{ \ \
}\mathtt{Re}\left(\zeta\right)<0,$ (35)

as well as the IWOP technique, we can obtain the explicit normally ordered
expansion of $S_{3}$:

$\displaystyle S_{3}$ $\displaystyle=\frac{1}{\cosh
r}\exp[-a_{1}^{{\dagger}}(a_{2}^{{\dagger}}\cos\theta+a_{3}^{{\dagger}}\sin\theta)\tanh
r]$ $\displaystyle\times\colon\exp[\frac{1-\cosh r}{\cosh
r}(a_{1}^{{\dagger}}a_{1}+a_{2}^{{\dagger}}a_{2}\cos^{2}\theta$
$\displaystyle+a_{3}^{{\dagger}}a_{3}\sin^{2}\theta+\frac{1}{2}a_{2}a_{3}^{{\dagger}}\sin
2\theta+\frac{1}{2}a_{2}^{{\dagger}}a_{3}\sin 2\theta)]\colon$
$\displaystyle\times\exp[a_{1}(a_{2}\cos\theta+\allowbreak
a_{1}\sin\theta)\tanh r].$ (36)

##  5 Wigner function of $\left\|000\right\rangle$

Wigner distribution function of quantum states [16, 17, 18] is widely studied
in quantum statistics and quantum optics and is very important tool for a
global description of nonclassical effect in the quantum system, which can be
measured by various means such as photon counting experiment and homodyne
tomography. Now we derive the Wigner function of $\left\|000\right\rangle$ by
using a new method.

Recalling that in Ref. [14] we have introduced the Weyl ordering form of
single-mode Wigner operator $\Delta_{1}\left(q_{1},p_{1}\right)$,

$\Delta_{1}\left(q_{1},p_{1}\right)=\genfrac{}{}{0.0pt}{}{:}{:}\delta\left(q_{1}-Q_{1}\right)\delta\left(p_{1}-P_{1}\right)\genfrac{}{}{0.0pt}{}{:}{:},$
(37)

where the symbols$\genfrac{}{}{0.0pt}{}{:}{:}\genfrac{}{}{0.0pt}{}{:}{:}$
denote the Weyl ordering, while its normal ordering form is

$\Delta_{1}\left(q_{1},p_{1}\right)=\frac{1}{\pi}\colon\exp\left[-\left(q_{1}-Q_{1}\right)^{2}-\left(p_{1}-P_{1}\right)^{2}\right]\colon.$
(38)

Thus the Wigner function for $\left|0\right\rangle$ can be easily expressed as
$\left\langle
0\right|\Delta_{1}\left(q_{1},p_{1}\right)\left|0\right\rangle=\frac{1}{\pi}\exp(-q_{1}^{2}-p_{1}^{2})$.
Note that the order of Bose operators $a_{1}$ and $a_{1}^{\dagger}$ within a
normally ordered product (or a Weyl ordered product) can be permuted. That is
to say, even though $[a_{1},a_{1}^{\dagger}]=1$, we can have $\colon
a_{1}a_{1}^{\dagger}\colon=\colon a_{1}^{\dagger}a_{1}\colon$
and$\genfrac{}{}{0.0pt}{}{:}{:}a_{1}a_{1}^{\dagger}\genfrac{}{}{0.0pt}{}{:}{:}=\genfrac{}{}{0.0pt}{}{:}{:}a_{1}^{\dagger}a_{1}\genfrac{}{}{0.0pt}{}{:}{:}.$
The Weyl ordering of operators has a remarkable property, i.e., the Weyl-
ordering invariance of operators under similar transformations, which means

$U\genfrac{}{}{0.0pt}{}{:}{:}\left(\circ\circ\circ\right)\genfrac{}{}{0.0pt}{}{:}{:}U^{-1}=\genfrac{}{}{0.0pt}{}{:}{:}U\left(\circ\circ\circ\right)U^{-1}\genfrac{}{}{0.0pt}{}{:}{:},$
(39)

as if the “fence” $\genfrac{}{}{0.0pt}{}{:}{:}\genfrac{}{}{0.0pt}{}{:}{:}$did
not exist when $U$ operates.

For 3-mode case, the Weyl ordering form of the Wigner operator is

$\Delta_{3}\left(\mathbf{q},\mathbf{p}\right)=\genfrac{}{}{0.0pt}{}{:}{:}\delta\left(\mathbf{q}-\mathbf{Q}\right)\delta\left(\mathbf{p}-\mathbf{P}\right)\genfrac{}{}{0.0pt}{}{:}{:},$
(40)

where $\mathbf{Q}=(Q_{1},Q_{2},Q_{3})^{T}$,
$\mathbf{P}=(P_{1},P_{2},P_{3})^{T}$, $\mathbf{q}=(q_{1},q_{2},q_{3})^{T}$ and
$\mathbf{p}=(p_{1},p_{2},p_{3})^{T}$. Then according to the Weyl ordering
invariance under similar transformations and using Eq.(14), we have

$\displaystyle S_{3}^{-1}\Delta_{3}\left(\mathbf{q},\mathbf{p}\right)S_{3}$
$\displaystyle=S_{3}^{-1}\genfrac{}{}{0.0pt}{}{:}{:}\delta\left(\mathbf{q}-\mathbf{Q}\right)\delta\left(\mathbf{p}-\mathbf{P}\right)\genfrac{}{}{0.0pt}{}{:}{:}S_{3}$
$\displaystyle=\genfrac{}{}{0.0pt}{}{:}{:}\delta\left(q_{k}-(e^{-\Lambda})_{ki}Q_{i}\right)\delta\left(p_{k}-(e^{\Lambda})_{ki}P_{i}\right)\genfrac{}{}{0.0pt}{}{:}{:}$
$\displaystyle=\genfrac{}{}{0.0pt}{}{:}{:}\delta\left((e^{\Lambda})_{ki}q_{i}-Q_{k}\right)\delta\left((e^{-\Lambda})_{ki}p_{i}-P_{k}\right)\genfrac{}{}{0.0pt}{}{:}{:}$
$\displaystyle=\genfrac{}{}{0.0pt}{}{:}{:}\delta\left(\mathbf{q}^{\prime}-\mathbf{Q}\right)\delta\left(\mathbf{p}^{\prime}-\mathbf{P}\right)\genfrac{}{}{0.0pt}{}{:}{:}$
$\displaystyle=\Delta_{3}\left(\mathbf{q}^{\prime},\mathbf{p}^{\prime}\right),$
(41)

where $q_{k}^{\prime}=(e^{\Lambda})_{ki}q_{i},$
$p_{k}^{\prime}=(e^{-\Lambda})_{ki}p_{i}$. Thus the Wigner function of
$\left\|000\right\rangle$ is

$\displaystyle\left\langle
000\right|S_{3}^{-1}\Delta_{3}\left(\mathbf{q},\mathbf{p}\right)S_{3}\left|000\right\rangle$
$\displaystyle=\left\langle
000\right|\Delta_{3}\left(\mathbf{q}^{\prime},\mathbf{p}^{\prime}\right)\left|000\right\rangle$
$\displaystyle=\frac{1}{\pi^{3}}\exp\left(-\mathbf{q}^{T}e^{2\Lambda}\mathbf{q}-\mathbf{p}^{T}e^{-2\Lambda}\mathbf{p}\right),$
(42)

where $e^{2\Lambda}$ and $e^{-2\Lambda}$ are given by $e^{\Lambda}$ in Eq.(10)
and $e^{-\Lambda}$ in Eq.(11), respectively.

In summary, we have shown that the operator
$S_{3}\equiv\exp[\mu(a_{1}a_{2}-a_{1}^{\dagger}a_{2}^{\dagger})+\nu(a_{1}a_{3}-a_{1}^{\dagger}a_{3}^{\dagger})]$
is a new 3-mode squeezed operator by calculating the quantum fluctuation for
3-mode quadratures. We have obtained the new 3-mode squeezed vacuum state and
derived the normally ordered expansion of $S_{3}$. The IWOP technique brings
convenience in our derivation.

## References

  * [1] Nielsen M. A. and Chuang I. L., Quantum Computation and Quantum Information (Cambridge University Press) 2000.
  * [2] Bouwmeester D. et al., The Physics of Quantum Information (Springer, Berlin) 2000.
  * [3] Fan H.-y., and Yu G.-c. Phys. Rev. A 65 (2002) 033829.
  * [4] Fan H.-y. and Klauder J. R., Phys. Rev. A, 49 (1994) 704.
  * [5] Mandel L. and Wolf E., Optical Coherence and Quantum Optics (Cambridge:Cambridge University Press) 1995.
  * [6] Loudon R. and Knight P. L., J. Mod. Opt., 34 (1987) 709.
  * [7] Dodonov V. V., J. Opt. B: Quantum Semiclass. Opt., 4 (2002) R1.
  * [8] Fan H.-y., Europhys. Lett., 23 (1993) 1.
  * [9] Fan H.-y., Europhys. Lett., 17 (1992) 285; 19 (1992) 443.
  * [10] Hu L.-y. and Fan H.-y.,Europhys. Lett., 85 (2009) 60001.
  * [11] Fan H.-y., J. Opt. B: Quantum Semiclass. Opt., 5 (2003) R147.
  * [12] Fan H.-y., J. Phys. A, 25 (1992) 3443; Fan H.-y. and Fan Y., Int. J. Mod. Phys. A, 17 (2002) 701.
  * [13] Fan H.-y., Mod. Phys. Lett. A, 15 (2000) 2297.
  * [14] Fan H.-y., Ann. Phys. (N.Y.), 323 (2008) 500; 1502.
  * [15] El-Orany F. A. A., et al., Opt. Commu. 283 (2010) 3158
  * [16] Wigner E. P., Phys. Rev., 40 (1932) 749.
  * [17] O’Connell R. F. and Wigner E. P., Phys. Lett. A, 83 (1981) 145.
  * [18] Schleich W., Quantum Optics (Wiley, New York) 2001.

Figure 1: (Colour online) The quantity $(\triangle X_{1})^{2}$ and $(\triangle
X_{2})^{2}$ as the function of squeezing parameter $\mu$ for different case
$\nu=0$ and $\nu=0.5$. Figure 2: (Colour online) The uncertainty value
$(\triangle X_{1})(\triangle X_{2})$ as the function of $r$ for different
$\theta$.