# Optomechanical Backaction in the Bistable Regime

L. F. Deeg<EMAIL_ADDRESS>Institute for Quantum Optics and Quantum
Information, Austrian Academy of Sciences, 6020 Innsbruck, Austria  Institute
for Experimental Physics, University of Innsbruck, 6020 Innsbruck, Austria  D.
Zoepfl  Institute for Quantum Optics and Quantum Information, Austrian Academy
of Sciences, 6020 Innsbruck, Austria  Institute for Experimental Physics,
University of Innsbruck, 6020 Innsbruck, Austria  N. Diaz-Naufal  Dahlem
Center for Complex Quantum Systems and Fachbereich Physik, Freie Universität
Berlin, 14195 Berlin, Germany  M. L. Juan  Institut Quantique and Département
de Physique, Université de Sherbrooke, Sherbrooke, Québec, J1K 2R1, Canada  A.
Metelmann  Dahlem Center for Complex Quantum Systems and Fachbereich Physik,
Freie Universität Berlin, 14195 Berlin, Germany  Institute for Theory of
Condensed Matter and Institute for Quantum Materials and Technology, Karlsruhe
Institute of Technology, 76131 Karlsruhe, Germany  Institut de Science et
d’Ingénierie Supramoléculaires (ISIS, UMR7006), University of Strasbourg and
CNRS  G. Kirchmair<EMAIL_ADDRESS>Institute for Quantum Optics
and Quantum Information, Austrian Academy of Sciences, 6020 Innsbruck, Austria
Institute for Experimental Physics, University of Innsbruck, 6020 Innsbruck,
Austria

###### Abstract

With a variety of realisations, optomechanics utilizes its light matter
interaction to test fundamental physics. By coupling the phonons of a
mechanical resonator to the photons in a high quality cavity, control of
increasingly macroscopic objects has become feasible. In such systems, state
manipulation of the mechanical mode is achieved by driving the cavity. To be
able to achieve high drive powers the system is typically designed such that
it remains in a linear response regime when driven. A nonlinear response and
especially bistability in a driven cavity is often considered detrimentally to
cooling and state preparation in optomechanical systems and is avoided in
experiments. Here we show, that with an intrinsic nonlinear cavity backaction
cooling of a mechanical resonator is feasible operating deeply within the
nonlinear regime of the cavity. With our theory taking the nonlinearity into
account, precise predictions on backaction cooling can be achieved even with a
cavity beyond the bifurcation point, where the cavity photon number spectrum
starts to deviate from a typical Lorentzian shape.

Preparing a mechanical mode near its motional groundstate has become a crucial
task for fundamental physics experiments and quantum sensing applications [1].
However, even at cryogenic temperatures massive mechanical resonators reside
in highly populated phonon states. This makes additional cooling necessary to
bring them towards their motional groundstate. Utilizing the interaction
between the photon mode of a cavity and the phonon mode of a mechanical
resonator, cooling of the mechanics can be achieved by feedback [2, 3, 4] and
sideband cooling [5, 6]. For the latter, cooling can be achieved when driving
the cavity below the resonance frequency, where we have to distinguish two
cases. In the first case, the setup resides within the bad cavity regime when
the cavity linewidth exceeds the mechanical frequency ($\kappa>\omega_{m}$).
There the achievable cooling is limited by unwanted backaction heating [1]. In
the opposite case - the so-called resolved sideband regime
($\kappa<\omega_{m}$) - ground state cooling can be reached as shown for
various systems e.g. systems based on microwave cavities [7, 8].

However, at high photon numbers such systems feature a nonlinear behaviour
causing the linearisation of the standard optomechanical theory to fail at
describing the optomechanical interaction [5, 9]. This nonlinear behaviour can
originate from an intrinsic nonlinearity in the cavity design, but is also a
fundamental effect of the optomechanical coupling scheme [10]. Thus, by
increasing the driving strength, optomechanical setups inevitably enter a
regime where the nonlinearity becomes relevant. For backaction cooling,
systems residing in the linear regime are commonly considered to be ideal,
since they allow for high photon numbers and predictable cooling with
negligible nonlinear properties. Thus, optomechanical experiments are
typically designed as linear as possible and drive strengths are limited to
avoid entering the nonlinear regime [11, 12].

In recent measurements [13] we showed that the intrinsic nonlinearity of the
cavity can be harnessed to increase cooling capabilities in the bad cavity
regime ($\kappa>\omega_{m}$) [10, 14, 15]. In this work, we deliberately
increase the drive strength above the critical input power of the cavity.
There the system undergoes a bifurcation from a monostable to a bistable
regime, where the cavity resides either in the low or high photon number
branch within a specific range of frequencies. We report on the observation of
backaction cooling even driving beyond the critical input power of the cavity.
The developed theory describes the observed backaction in the bistable regime
depending on the photon state of the cavity.

The Hamiltonian for a nonlinear cavity dispersively coupled to a mechanical
resonator in the rotating frame of the drive can be written as [10, 15]

$\hat{\mathcal{H}}/\hbar=-\Delta\hat{a}^{\dagger}\hat{a}-\frac{\mathcal{K}}{2}\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}+\omega_{m}\hat{b}^{\dagger}\hat{b}+g_{0}\hat{a}^{\dagger}\hat{a}(\hat{b}+\hat{b}^{\dagger})+\hat{\mathcal{H}}_{\mathrm{d}}$
(1)

with $\Delta=\omega_{d}-\omega_{c}$ the drive ($\omega_{d}$) - cavity
($\omega_{c}$) detuning, $\omega_{m}$ the mechanical resonance frequency,
$\hat{a}\,(\hat{b})$ and $\hat{a}^{\dagger}\,(\hat{b}^{\dagger})$ the
annihilation and creation operators of the cavity (mechanics). The Kerr-
constant given by $\mathcal{K}$ represents the lowest order nonlinearity of
the cavity causing a photon number dependent frequency shift. Here, $g_{0}$ is
the single photon optomechanical coupling strength and
$\hat{\mathcal{H}}_{\mathrm{d}}=\alpha_{d}\hat{a}^{\dagger}+h.c.$ represents
the time independent external drive. With utilizing standard input-output
theory the equation of motion of the cavity can be derived as [16]

$\frac{d}{dt}\hat{a}=-i[\hat{a},\hat{\mathcal{H}}]-\frac{\kappa}{2}\hat{a}-\sqrt{\kappa_{c}}\hat{a}_{\mathrm{in}}.$
(2)

Here, the total linewidth $\kappa=\kappa_{c}+\kappa_{i}$ is introduced as the
sum of an intrinsic loss rate $\kappa_{i}$ and a coupling rate $\kappa_{c}$ to
an input photon field $\hat{a}_{\mathrm{in}}$ resulting in an input photon
number $n_{\mathrm{in}}$.

Figure 1: a) Photon number spectrum $g_{0}^{2}S_{\mathrm{nn}}[\omega]$ for
cavities with $\mathcal{K}_{0}/2\pi=0\,\mathrm{kHz}$,
$\mathcal{K}_{1}/2\pi=8\,\mathrm{kHz}$ and
$\mathcal{K}_{2}/2\pi=16\,\mathrm{kHz}$ and a coupling strength of
$g_{0}/2\pi=1.7\,\mathrm{kHz}$. The drive is detuned by
$\Delta/\omega_{m}=-11$ from the bare frequency of the cavity $\omega_{c}$ and
set to a constant power $n_{in}/n_{bi}=1.5$ with respect to $\mathcal{K}_{2}$.
Grey dashed lines indicate the (Anti-)/Stokes scattering processes. The photon
induced frequency shift due to $\mathcal{K}$ is evident and becomes more
pronounced with increasing $\mathcal{K}$. For $\mathcal{K}_{2}$,
$S_{\mathrm{nn}}[\omega]$ shows two stable solutions indicating the
bifurcation due to the splitting of the photon number $n_{c}$ shown in the
inset. b) Imbalance of the (Anti-)/Stoke rates for different detunings
$\Delta/\omega_{m}$. The larger imbalance shows the enhanced cooling
capability for a nonlinear cavity in the mono and bistable regime compared to
a linear system. For $\mathcal{K}_{2}$ both solutions feature a discontinuity
close to switching to the other branch.

Backaction cooling in the unresolved regime can be explained via the
conventional scattering framework taking into account the shape of the photon
number spectrum
$S_{\mathrm{nn}}[\omega]=\int_{-\infty}^{\infty}\,dt\,e^{i\omega
t}\,\langle(\hat{a}^{\dagger}\hat{a})(t)(\hat{a}^{\dagger}\hat{a})(0)\rangle$
within the rotating frame of the drive [10, 17]. Due to the optomechanical
coupling, photons of an input drive within the cavity resonance are scattered
to higher/lower frequencies by annihilation/creation of a phonon in the
resonator. These (Anti-)Stokes processes occur at a rate
$(\Gamma_{AS})\,\Gamma_{S}$ and are directly proportional to
$g_{0}^{2}S_{\mathrm{nn}}[\omega]$ of the cavity at
$\omega=\omega_{d}\pm\omega_{m}$ as indicated in Fig. 1a). Here, the spectra
$S_{\mathrm{nn}}[\omega]$ of a linear ($\mathcal{K}=0$) and two nonlinear
($\mathcal{K}>0$) systems are depicted. For a drive detuned by
$\Delta/\omega_{m}=-11$, $S_{\mathrm{nn}}[\omega]$ of the linear system has a
Lorentzian shape and is symmetric around the resonance of an undriven system.
On the contrary, for a nonlinear system ($\mathcal{K}>0$) in presence of a
strong drive $S_{\mathrm{nn}}[\omega]$ becomes asymmetric and the resonance
frequency shifts. The power dependent shift is a consequence of the Kerr-term
in Eq. 1 influencing $S_{\mathrm{nn}}[\omega]$. If the drive exceeds the
critical input power given by

$n_{bi}=\frac{\kappa^{2}}{3\sqrt{3}\mathcal{K}},$ (3)

the cavity photon number $n_{c}$ and thus $S_{\mathrm{nn}}[\omega]$ splits
into the low and high photon number branch within a certain frequency range
corresponding to the red and green curves in Fig. 1. At these high driving
strengths, $S_{\mathrm{nn}}[\omega]$ deviates from a purely Lorentzian shape
as it is visible for the red/green traces in Fig. 1a) [10]. Applying an off-
resonance drive tone to the cavity, $S_{\mathrm{nn}}[\omega]$ around the drive
tone leads to an imbalance in $(\Gamma_{AS})\,\Gamma_{S}$, which results in a
net cooling/heating of the phonon mode. In Fig. 1b) this imbalance of
$\Gamma_{S}-\Gamma_{AS}$ is tracked for different detunings $\Delta$ of the
drive tone with respect to the cavity resonance, where cooling occurs for
$\Gamma_{AS}>\Gamma_{S}$.

For a linear cavity, being red detuned $(\Delta<0)$ cooling and for blue
detunings $(\Delta>0)$ heating is observed. Since for a linear cavity
$S_{\mathrm{nn}}[\omega]$ is symmetric around the resonance frequency, also
the imbalance and the corresponding backaction on the mechanics is symmetric.
In a nonlinear but otherwise identical system, the imbalance
$\Gamma_{S}-\Gamma_{AS}$ is altered due to the asymmetric shape of
$S_{\mathrm{nn}}[\omega]$. This leads to an increase in the cooling capability
as illustrated in [10, 13] and shown in Fig. 1b). Within the bistable regime
of such a system, an abrupt change in the rates can be observed right at the
detuning where the photon number jumps from one to the other branch. A drive
close to these detunings leads to a suppression of $\Gamma_{S}$ compared to
$\Gamma_{AS}$. Thus, in Fig. 1b) a rapid increase in the cooling capability
within the proximity of the switching is obtained. However, to observe
backaction in the bistable regime, it is crucial to be sufficiently close to
these detunings. This makes cooling beyond bifurcation of the cavity
susceptible to frequency fluctuations, which can cause the cavity to switch
branches nullifying the observed backaction. Therefore, it is crucial to
establish a experimental setup well isolated from external noise sources, like
flux or vibrational noise, to avoid those switching events.

Figure 2: Setup and characterisation. a) Schematic circuit diagram of the
setup with junction inductance $L_{J}$, cavity inductance $L$ and $C$, $C_{c}$
and $C_{g}$ as the self, coupling and ground capacitance, respectively. b)
Picture of the sample. Silicon substrate in gold, u-shaped cavity in silver,
cantilever in grey and magnet false colored in purple (a,b taken from [13]).
c) Cavity response for different input powers. For intermediate powers the
effects of the nonlinearity become visible and for highest power the cavity
shows the splitting into low (red) and high (green) photon number branch. For
this power the arrows indicate the direction of the frequency sweep.

The experiment features a flipchip design of a microstrip $\lambda/2$-cavity
[18]coupled to a cantilever as shown in Fig. 2a) and b), similar to the setup
discussed in [19, 13]. To realise an inductive coupling between cavity and
mechanical resonator, a SQUID is embedded in the center of the cavity making
it flux sensitive, similar to other systems based on an inductive coupling
scheme [20, 21, 22]. By attaching a magnetic particle to the tip of the
cantilever its motion modulates the magnetic flux through the SQUID, which
couples the two oscillator modes. The chip is placed inside a 3D-waveguide
with a central coil around it, allowing us to apply an external DC magnetic
field for tuning purposes. The whole setup is mounted to the base of a
dilutional cryostat operated at $100\,\mathrm{mK}$. Additionally, to decouple
our system from any mechanical vibrations we implemented a vibration isolation
setup consisting of a long nylon wire and a stainless steel spring acting as a
lowpass filter with a resonance frequency of $\omega_{z}/2\pi\approx
2.4\,\mathrm{Hz}$ [23]. The cavity has a frequency of
$\omega_{c}/2\pi=8.1\,\mathrm{GHz}$, and a total linewidth of
$\kappa/2\pi=2.8\,\mathrm{MHz}$, whereas the cantilever has a frequency of
$\omega_{m}/2\pi=287.3\,\mathrm{kHz}$ and a linewidth of
$\Gamma_{m}/2\pi=0.4\,\mathrm{Hz}$.

The nonlinear behaviour of the cavity can be visualized by scanning its
response for different probe powers as shown in Fig. 2c). By increasing the
power we first observe a frequency shift of the cavity response accompanied by
an asymmetric shape of the resonance. Further increasing the drive power above
$n_{in,bi}$ one can see the splitting of the cavity response into the two
branches. For both branches the detuning, where the cavity switches into the
other branch, is visible by a discontinuous jump in the response. Depending on
the direction of the frequency sweep - represented by the arrows in Fig. 2c) -
the different states can be accessed.

Figure 3: a) Relaxation of the frequency shift $\delta\mspace{-4.0mu}f$ after
driving the cavity at different powers, for the cavity being initialised in
the high photon number branch. Solid lines represent an exponential fit to
extract the timescale $\overline{\tau}=(0.96\pm 0.02)\,\mathrm{s}$. Inset
shows a zoom in on the data plotted on a log-scale. b) Frequency shift
$\delta\mspace{-4.0mu}f$ against probe-cavity detuning $\Delta$ for a drive
strength $n_{in}/n_{bi}=3.0$. Probing the different branches by tuning the
probe towards the cavity from low/high frequencies to scan the low (red) and
high (green) branch.

Additionally to the Kerr induced frequency shift shown in Fig. 2d), we
observed another frequency shift $\delta\mspace{-4.0mu}f$ of the cavity. This
shift can be measured by weakly probing the cavity response in presence of a
strong drive tone. By populating the cavity, the resonance frequency shifts
until it reaches a new steady state on the timescale of seconds. With higher
drive strengths or placing the drive closer to the resonance - both
effectively increasing the intracavity photon number -
$\delta\mspace{-4.0mu}f$ between the new steady state $f_{ss}$ and initial
$f_{i}$ frequency grows. To quantify the timescale of this shift, we measure
the evolution of the cavity response after turning off the drive tone. As
shown in Fig. 3a) $\delta\mspace{-4.0mu}f(t)$ follows an exponential decay
back to its initial value. A fit to the data yields a slow relaxation time of
$\overline{\tau}=(0.96\pm 0.02)\,\mathrm{s}$ independent from the photon
number, whereas the amplitude of $\delta\mspace{-4.0mu}f$ significantly
depends on it.

We can use this photon number dependence of the frequency shift to determine
in which state the cavity ended up after applying a drive. Since the photon
number in the low and high photon number branch significantly differs,
$\delta\mspace{-4.0mu}f$ measured right after turning off the drive will be
large in the high and small for the low photon number branch. In Fig. 3b) this
is shown by displaying $\delta\mspace{-4.0mu}f=f_{ss}-f_{i}$ against cavity-
drive detuning $\Delta$ for a drive exceeding the critical input power. While
residing in the low photon number branch, the cavity is almost not populated
resulting in no detectable shift. However, close to switching to the high
photon number branch a small shift is observed followed by a distinct
frequency jump upon switching. Scanning the high state we observe a large
frequency shift due to the highly populated cavity until a detuning is
reached, where the cavity switches into the low photon number branch visible
by $\delta\mspace{-4.0mu}f\approx 0$. Between the switching events of the
branches, the cavity shows two distinct shifts associated to either one of the
cavity states. Although the origin of the observed shift remains unclear [23],
we can take it into account by adjusting the measurement procedure to contain
wait times after turning on the drive to ensure a steady state of our cavity.

To characterize the mechanics, we measure the homodyne noise spectrum of a
probe tone applied to the cavity. Due to the coupling of the cavity to the
mechanics the probe tone undergoes an effective amplitude (phase) modulation
at the mechanical frequency $\omega_{m}$ passing through the system [19]. To
extract the phonon occupation $\langle n_{m}\rangle$, mechanical frequency
$\omega_{m}$, and the linewidth $\Gamma_{m}$ we fit the model of a damped
harmonic oscillator to the resulting sidebands and calibrate the signal via a
method derived in [24]. By measuring the mechanical signature on the probe
tone for different drive-cavity detunings $\Delta=\omega_{d}-\omega_{c}$ the
backaction on the mechanics can be studied by tracking $\langle n_{m}\rangle$,
$\omega_{m}$ and $\Gamma_{m}$. To determine the influence of the intrinsic
cavity nonlinearity we take cooling traces for different input powers.
Furthermore, to ensure a steady state of the cavity we slowly tune the drive
in resonance in the same directions as shown in Fig. 2d) to select the
low/high photon number branch beyond bifurcation. Additionally, we utilize the
discussed properties of $\delta\mspace{-4.0mu}f$ to determine the cavity state
during the measurement to avoid data points containing a switching event.

Figure 4: Cooling traces at $g_{0}/2\pi=99\,\mathrm{Hz}$. a) Phonon number
$\langle n_{m}\rangle$, b) linewidth $\Gamma_{m}/2\pi$ and c) frequency shift
$\delta\omega_{m}/2\pi$ for intermediate powers $n_{in}/n_{bi}=0.5$. Grey
lines display a steady state at an effective bath temperature
$T_{\mathrm{eff}}=267\,\mathrm{mK}$. Phonon number $\langle n_{m}\rangle$,
linewidth $\Gamma_{m}/2\pi$ and frequency shift $\delta\omega_{m}/2\pi$ for
higher powers of $n_{in}/n_{bi}=1.9$ in d-f) and $n_{in}/n_{bi}=3.0$ in g-i),
respectively. Data and theoretical extrapolation from $n_{in}/n_{bi}=0.5$ in
red for the low and green for the high photon number branch. Filled circles
correspond to data taken with the pulsetube of the cryostat turned off to
further reduce vibrations in the system

Fig. 4a-c) shows a cooling trace at moderate coupling strength of
$g_{0}/2\pi=(99\pm 1)\,\mathrm{Hz}$ to minimize flux noise effects [23].
Fitting the data with a theory derived in [10, 13] we observe good agreement
with the experimental results. The fit yields an input photon number of
$n_{in}/n_{bi}=(0.54\pm 0.02)$ and a Kerr constant of $\mathcal{K}/2\pi=(14\pm
1)\,\mathrm{kHz}$. Already at these drive powers, the cooling trace shows the
effects of the nonlinearity, namely an asymmetric backaction trace with a
narrow cooling window. The displayed thermal phonon number (grey line)
corresponds to an extracted effective bath temperature of
$T_{\mathrm{eff}}=267\,\mathrm{mK}$ even though we set the base temperature of
the fridge to $100\,\mathrm{mK}$ [23].

Cooling traces for increased input powers corresponding to $n_{in}/n_{bi}=1.9$
and $n_{in}/n_{bi}=3.0$ are shown in Fig. 4d-f) and i-g), respectively. For
the different input powers, over a detuning range of $\sim 0.4\,\mathrm{MHz}$
and $\sim 2\,\mathrm{MHz}$ we observe two distinct backaction curves of the
mechanics depending on the selected branch. The solid lines in the plots
depict the theoretical prediction extrapolated from the parameters obtained
from the fit in Fig. 4a-c). Even though, we increased the input power by a
factor of $\sim 4$ and $\sim 6$ resulting in a cavity well beyond the
bifurcation point, the prediction accurately describes the data for the two
different branches. In both cases, the backaction in the low photon number
branch exhibits a narrow cooling feature up to the point at which the cavity
switches to the high photon number branch having reached the end of the
bifurcation region. Upon switching, the cavity resides in the high photon
number branch, where the photon number spectrum of the cavity results in
backaction heating. The obtained experimental data agrees well with the
predictions and the detuning at which branch switching is observed matches
with the theory.

In case of the high photon number branch we observe heating over a large
frequency range before the cavity state switches. The high photon number
branch shows a very narrow cooling feature that can not be experimentally
accessed. This is due to constant heating of the mechanical mode leading to a
highly excited state as we tune the probe tone slowly in from the blue side to
ensure staying in the high photon number branch. The corresponding large
motional amplitude leads to fluctuations of $\omega_{c}$, which results in the
cavity switching branches before reaching the cooling feature in the high
photon number branch. The obtained experimental results for the low and high
photon number branch demonstrate, that we have good control over the cavity
state in the bistable regime. Furthermore, the theory predictions match with
the data for all measured input powers [23], which underlines a good
understanding of the studied system.

Summarizing, we demonstrated that we can operate our system even beyond the
bifurcation of the cavity and expand the observation on backaction cooling for
nonlinear systems made in [13]. Due to the control over the selected branches
in this regime we achieved good agreement with the underlying theory. By
extrapolating fit results over a factor $6$ in input power we can precisely
predict the obtained cooling behaviour demonstrating a good understanding of
processes at hand. Furthermore, the observed cooling behaviour beyond the
bifurcation originates due to a deviation of the cavity photon number spectrum
from the typical Lorentzian shape. An interesting fact is that the best
cooling is theoretically and experimentally obtained in the low instead of the
high photon number branch, contrary to the common opinion that an increased
photon number should lead to an enhanced cooling. Overall, the resulting
cooling in the bistable regime shown here is limited by flux noise in the
system, preventing us from precisely accessing the narrow frequency range in
which best cooling is expected. Also, further investigations have to be made
to grasp the origin of the observed slow frequency shift of our cavity.
Nevertheless, the presented results demonstrate that state manipulation in
optomechanical systems should not be restricted only to the linear regime.
Exploiting the nonlinearities may offer novel mechanisms to be explored for
optomechanical systems in the unresolved sideband regime.

###### Acknowledgements.

We want to thank Hans Huebl and Christian Schneider for the fruitful
discussions. Further we thank our in-house workshop. L.D. is supported by the
Austrian Science Fund FWF within the DK-ALM (W1259-N27). N.D.N and A.M.
acknowledge funding by the project CRC 183. A.M. acknowledges funding by the
Deutsche Forschungsgemeinschaft through the Emmy Noether program (Grant No.
ME4863/1-1). M. L. J. acknowledges funding by the Canada First Research
Excellence Fund. This work was supported by the European Union‘s Horizon
Europe 2021-2027 Framework Programme under Grant Agreement No. 101080143
(SuperMeQ).

## References

  * Aspelmeyer  _et al._ [2014] M. Aspelmeyer, T. J. Kippenberg, and F. Marquardt, Cavity optomechanics, Reviews of Modern Physics 86, 1391 (2014).
  * Mancini  _et al._ [1998] S. Mancini, D. Vitali, and P. Tombesi, Optomechanical Cooling of a Macroscopic Oscillator by Homodyne Feedback, Physical Review Letters 80, 688 (1998).
  * Rossi  _et al._ [2018] M. Rossi, D. Mason, J. Chen, Y. Tsaturyan, and A. Schliesser, Measurement-based quantum control of mechanical motion, Nature 563, 53 (2018).
  * Tebbenjohanns  _et al._ [2020] F. Tebbenjohanns, M. Frimmer, V. Jain, D. Windey, and L. Novotny, Motional Sideband Asymmetry of a Nanoparticle Optically Levitated in Free Space, Physical Review Letters 124, 013603 (2020).
  * Marquardt  _et al._ [2007] F. Marquardt, J. P. Chen, A. A. Clerk, and S. M. Girvin, Quantum Theory of Cavity-Assisted Sideband Cooling of Mechanical Motion, Physical Review Letters 99 (2007).
  * Wilson-Rae  _et al._ [2007] I. Wilson-Rae, N. Nooshi, W. Zwerger, and T. J. Kippenberg, Theory of Ground State Cooling of a Mechanical Oscillator Using Dynamical Backaction, Physical Review Letters 99, 093901 (2007).
  * Teufel  _et al._ [2011] J. D. Teufel, T. Donner, D. Li, J. W. Harlow, M. S. Allman, K. Cicak, A. J. Sirois, J. D. Whittaker, K. W. Lehnert, and R. W. Simmonds, Sideband cooling of micromechanical motion to the quantum ground state, Nature 475, 359 (2011).
  * Chan  _et al._ [2011] J. Chan, T. P. M. Alegre, A. H. Safavi-Naeini, J. T. Hill, A. Krause, S. Gröblacher, M. Aspelmeyer, and O. Painter, Laser cooling of a nanomechanical oscillator into its quantum ground state, Nature 478, 89 (2011).
  * Safavi-Naeini  _et al._ [2013] A. H. Safavi-Naeini, J. Chan, J. T. Hill, S. Gröblacher, J. Miao, Y. Chen, A. Aspelmeyer, and O. Painter, Laser noise in cavity-optomechanical cooling and thermometry, New Journal of Physics 15, 035007 (2013).
  * [10] N. Diaz-Naufal, D. Zoepfl, M. L. Juan, C. M. F. Schneider, L. F. Deeg, G. Kirchmair, and A. Metelmann, In preparation .
  * Teufel  _et al._ [2008] J. D. Teufel, J. W. Harlow, C. A. Regal, and K. W. Lehnert, Dynamical backaction of microwave fields on a nanomechanical oscillator, Phys. Rev. Lett. 101, 197203 (2008).
  * Peterson  _et al._ [2019] G. A. Peterson, S. Kotler, F. Lecocq, K. Cicak, X. Y. Jin, R. W. Simmonds, J. Aumentado, and J. D. Teufel, Ultrastrong parametric coupling between a superconducting cavity and a mechanical resonator, Phys. Rev. Lett. 123, 247701 (2019).
  * Zoepfl  _et al._ [2023] D. Zoepfl, M. L. Juan, N. Diaz-Naufal, C. M. F. Schneider, L. F. Deeg, A. Sharafiev, A. Metelmann, and G. Kirchmair, Kerr enhanced backaction cooling in magnetomechanics, Phys. Rev. Lett. 130, 033601 (2023).
  * Laflamme and Clerk [2011] C. Laflamme and A. A. Clerk, Quantum-limited amplification with a nonlinear cavity detector, Physical Review A 83, 033803 (2011).
  * Nation  _et al._ [2008] P. D. Nation, M. P. Blencowe, and E. Buks, Quantum analysis of a nonlinear microwave cavity-embedded dc SQUID displacement detector, Physical Review B 78, 104516 (2008).
  * Gardiner and Collett [1985] C. W. Gardiner and M. J. Collett, Input and output in damped quantum systems: Quantum stochastic differential equations and the master equation, Physical Review A 31, 3761 (1985).
  * Marquardt  _et al._ [2008] F. Marquardt, A. A. Clerk, and S. M. Girvin, Quantum theory of optomechanical cooling, Journal of Modern Optics 55, 3329 (2008).
  * Zoepfl  _et al._ [2017] D. Zoepfl, P. R. Muppalla, C. M. F. Schneider, S. Kasemann, S. Partel, and G. Kirchmair, Characterization of low loss microstrip resonators as a building block for circuit QED in a 3D waveguide, AIP Advances 7, 085118 (2017).
  * Zoepfl  _et al._ [2020] D. Zoepfl, M. L. Juan, C. M. F. Schneider, and G. Kirchmair, Single-Photon Cooling in Microwave Magnetomechanics, Physical Review Letters 125, 023601 (2020).
  * Bothner  _et al._ [2022] D. Bothner, I. C. Rodrigues, and G. A. Steele, Four-wave-cooling to the single phonon level in Kerr optomechanics, Communications Physics 5, 33 (2022).
  * Luschmann  _et al._ [2022] T. Luschmann, P. Schmidt, F. Deppe, A. Marx, A. Sanchez, R. Gross, and H. Huebl, Mechanical frequency control in inductively coupled electromechanical systems, Scientific Reports 12, 1608 (2022).
  * Bera  _et al._ [2021] T. Bera, S. Majumder, S. K. Sahu, and V. Singh, Large flux-mediated coupling in hybrid electromechanical system with a transmon qubit, Communications Physics 4, 12 (2021).
  * [23] See supplementary material at [URL] which includes Refs. [5], [22-25], [27-29], [33-34] and [42-49] for additional information on the setup, the nonlinear cooling theory, the data analysis, the flux noise model, verification of the coupling strength calibration and supplementary measurement data.
  * Gorodetksy  _et al._ [2010] M. L. Gorodetksy, A. Schliesser, G. Anetsberger, S. Deleglise, and T. J. Kippenberg, Determination of the vacuum optomechanical coupling rate using frequency noise calibration, Optics Express 18, 23236 (2010).
  * Probst  _et al._ [2015] S. Probst, F. B. Song, P. A. Bushev, A. V. Ustinov, and M. Weides, Efficient and robust analysis of complex scattering data under noise in microwave resonators, Review of Scientific Instruments 86, 024706 (2015).
  * Seis  _et al._ [2022] Y. Seis, T. Capelle, E. Langman, _et al._ , Ground state cooling of an ultracoherent electromechanical system, Nat Commun 13 1507 (2022).

##  Appendix S. 1 Setup

The sample as well as the measurement setup (Fig. 2 and Fig. S. 1) is very
similar to the one described in [19, 13]. To decouple our system from
vibrations of the fridge we implemented a one stage suspension setup
consisting of a nylon wire and a steel spring. To achieve reasonable damping
in the xyz-plane the setup is suspended from the still plate as shown in Fig.
S. 2a). To keep the vibrations coupled in over the in-/output lines low we
used cryoflex cables from Delft circuits and to ensure thermalisation of the
setup we used 3 flexible OFHC-copper braids.

Previously suffering from vibrational noise at
$1\,\mathrm{Hz}\,-\,1\,\mathrm{kHz}$ [13] with running pulse tube cooler, the
suspension setup decouples our system significantly from this frequency range
as shown in Fig. S. 2b). To characterise the isolation we shortly turned on
the pulse tube cooler at room temperature and measured the mechanical
vibrations with an accelerometer attached to the setup, either rigidly
connected to the base or suspended from the still plate. With the suspension
we achieve a damping of the vibrational noise in the low frequency range up to
$1\,\mathrm{kHz}$ by $\sim 20\,\mathrm{dBm}$. Above $1\,\mathrm{kHz}$ the
detection is limited by the cable connected to the accelerometer. Furthermore,
to prevent a ring up scenario of the suspended setup at its resonant
frequency, we implemented an eddy current damping tube - not shown in Fig. S.
2a) as it was implemented later. Fig. S. 2c) depicts a mechanical spectrum of
the cantilever at $150\,\mathrm{mK}$ measured with the pulse tube turned on
and off. Since both spectra match and the obtained occupation in both cases is
close to the thermal occupation of $n^{\mathrm{th}}_{\mathrm{m}}=1.1\cdot
10^{4}$, the suspension setup allows us to measure the mechanical signal with
the pulse tube running, which was previously not possible. This leads to an
increase in the data acquisition rate as well as allowing us to measure the
data for the backaction beyond bistability due to the increased stability.

Figure S. 1: Schematic drawing of the setup. The dashed circle indicates the
positions of the eddy current damping tube not shown in Fig. S. 2a). The
spring is connected to the setup via a tower structure inside a brass tube
attached to the base plate of the cryostat. For the damping we then attach
several NdFeB magnets (grey blocks in the zoom in). Figure S. 2: a) Photograph
of the suspended setup with the spring and the wire highlighted. At the bottom
the beginning of the magnetic shield is visible. b) Room temperature
accelerometer measurements to compare the vibrational noise for a system
rigidly connected to the base plate of the cryostat and one suspended from the
still plate. Above $1\,\mathrm{kHz}$ the detection is limited by the cable
connected to the accelerometer. c) Mechanical spectrum of the resonator at
$T_{\mathrm{bath}}=150\,\mathrm{mK}$ measured with running pulsetube and
pulsetube turned off for a rigidly connected and suspended setup.

##  Appendix S. 2 Data evaluation and Treatment

###  S. 2.1 Cavity

The cavity parameters can be extracted using the circle fit routine for a
response measured with a VNA. The transmission $S_{21}$ in a hanger-notch-
configuration is then given by [25]

$S_{21}(\omega)=ae^{i\alpha}e^{-i\tau\omega}\left(1-\frac{Q_{l}/|Q_{c}|e^{i\phi_{0}}}{1+2iQ_{l}\frac{\omega-\omega_{c}}{\omega}}\right)$
(S. 1)

Within the parenthesis is the ideal response of a resonator in transmission
with the loaded $Q_{l}$ and coupled quality factor $Q_{c}$, the impedance
missmatch $\phi_{0}$ and the resonance frequency $\omega_{c}$. Environmental
effects are taken into account by $a,\alpha$ and $\tau$ quantifies the
electric delay.

Figure S. 3: a) and b) Cavity response with increasing drive strength scanning
the low and high branch, respectively. c) Amplitude data for 3 different
powers, where for highest powers the signal bifurcates. Low and high branch
accessed by scanning the drive from different directions indicated by the
arrows. d) Data in the complex plane.

A good estimate for the Kerr-constant can be achieved by adapting the circle
fit routine by adding the photon number induced cavity shift of a nonlinear
cavity. For that the resonance frequency $\omega_{c}$ in Eq. S. 1 has to be
transformed to

$\omega_{c}\to\,\omega_{c}-\mathcal{K}n_{c}$ (S. 2)
$n_{c}[(\omega_{c}-\mathcal{K}n-\omega)^{2}+(\frac{\omega_{c}}{2Q_{l}})^{2}]=\frac{\omega_{c}}{2|Q_{c}|}\frac{P_{g}}{\hbar\omega}$
(S. 3)

with the Kerr-constant $\mathcal{K}$, the intracavity photon number $n_{c}$
and the drive power $P_{g}$ reaching the cavity. The intracavity photon number
for a nonlinear system is given by the cubic equation in Eq. S. 3 [10]. To be
compatible with the input-output theory convention leading to the $S_{21}$
response given in Eq. S. 1 a factor $1/2$ has to be added for the photon
input. By taking the input attenuation into account one can extract the Kerr-
constant by fitting consecutive cavity responses with increasing power Fig. S.
3. The fit yields a Kerr-constant of $\mathcal{K}=(12\pm
4)\,\mathrm{KHz/photon}$ being in good agreement with the results observed by
fitting the cooling traces in the main text. The given error results from the
uncertainty of the input attenuation, assumed to be $\pm 2\,\mathrm{dB}$.

###  S. 2.2 Mechanics

The measurements and data evaluation for the mechanical data is not
significantly changed from the procedure presented in the Appendix of [13].
The only addition to the measurements was the slow tune in of the drive tone
before measuring the mechanical spectra and a fast VNA trace triggered by
turning off the drive to determine the frequency shift
$\delta\mspace{-4.0mu}f$. For the data evaluation beyond bistability the
spectra were additionally sorted corresponding to the state of the cavity
based on the $\delta\mspace{-4.0mu}f$ before using the binning method.

##  Appendix S. 3 Temperature scan

Using the heterodyne detection scheme proposed by [24] we only have access to
$g_{0}^{2}\langle n_{m}\rangle$ by measuring the area below the mechanical
signal. Thus, it is essential to know our coupling strength $g_{0}$ at a given
bias frequency to be able to determine $\langle n_{m}\rangle$ for backaction
measurements. By assuming, that the mechanical mode thermalises with the base
temperature $T$ of the cryostat the phonon occupation is given by the
Boltzmann occupation $\langle
n^{th}_{m}\rangle\approx\frac{k_{B}T}{\hbar\omega_{m}}$. Therefore, measuring
$g_{0}^{2}\langle n^{th}_{m}\rangle$ for different temperatures $T$ allows us
to extract the single photon coupling strength $g_{0}$ at a given bias
frequency. To ensure that the mechanical mode is in thermal equilibrium we
measure the mode by using a weak probe tone tuned on resonance to avoid
backaction. The results are shown in Fig. S. 4.

For temperatures below $250\,\mathrm{mK}$ the data indicates that the
mechanical mode does not thermalise to the base temperature, similar to
observations for other mechanical systems at low temperature [26]. However,
for temperatures above $250\,\mathrm{mK}$ $g_{0}^{2}\langle n^{th}_{m}\rangle$
follows a linear trend allowing us to extract $g_{0}=99\pm 1\,\mathrm{Hz}$,
being consistent with previous measurements [19, 13] where the cantilever
thermalised. Since the coupling strength at this bias frequency should not
change with temperature, we can extract an effective bath temperature of the
mechanical mode to be $T_{\mathrm{eff}}=267\,\mathrm{mK}$ at a base
temperature of $T=100\,\mathrm{mK}$. The observed baseline of the cooling
traces for different drive powers agrees with the extracted
$T_{\mathrm{eff}}$, making the assumption of an effectively increased bath
temperature of the mechanics reasonable. Furthermore, the observed linewidth
follows a temperature dependence similar to previous measurements in [19, 13]
with $\Gamma_{m}/2\pi=0.4\,\mathrm{Hz}$ at $T=100\,\mathrm{mK}$. This further
reinforces the assumption of an effective bath temperature acting as an
additional phonon reservoir for our resonator.

Figure S. 4: a)Mechanical occupation $g_{0}^{2}n_{m}$ measured at different
base temperatures $T_{\mathrm{base}}$ with the cavity bias frequency used in
the main text. Colored data points used to calibrate $g_{0}$. Grey data points
indicate, that the mechanical mode does not thermalise to the base
temperature. Horizontal line represents the effective bath temperature
$T_{\mathrm{eff}}$ at $T=100\,\mathrm{mK}$. b) The mechanical linewidth
against base temperature.

##  Appendix S. 4 Frequency shift $\delta\mspace{-4.0mu}f$

Figure S. 5: a) relaxation of the frequency shift $\delta\mspace{-4.0mu}f$ for
different drive powers at $T_{\mathrm{base}}=100\,\mathrm{mK}$, b) at
different base temperature for the same drive power $n_{in}/n_{bi}=3.0$. c)
Comparison of the relaxation process measured on the left side - as in the
main text - and right side of the fluxmap at
$T_{\mathrm{base}}=50\,\mathrm{mK}$ and $n_{in}/n_{bi}\approx 1.0$. d) Reached
steady state frequency at the same power as in c) against set detuning
$\Delta/2\pi$ while probing the low and high photon number branch for left and
right side of the fluxmap. Data above the black line correspond to a positive
and below to a negative shift.

As mentioned in the main text, we observed a frequency shift
$\delta\mspace{-4.0mu}f$ of the cavity on time scales of seconds when strongly
driven. Populating the cavity causes the resonance frequency to shift reaching
a new steady state frequency, where the absolute shift increases with cavity
population. To quantify the timescale, we measure the relaxation process
towards the initial frequency after turning off the applied drive by weakly
probing the cavity. In Fig. S. 5a) the dependence of this relaxation process
is shown for different drive powers. Since only the absolute shift and not the
timescale changes with power, we can use this behaviour to distinguish if the
cavity resided in the low or high photon number branch during backaction
measurements as stated in the main text.

Furthermore, the observed frequency shift also prevents continuously switching
between the states during a measurement for detunings within the bistable
range. Since the cavity shifts towards lower frequencies with increasing
intracavity photon numbers, switching from the low to the high photon number
branch causes the cavity to shift, effectively increasing the detuning between
the drive and frequency associated to the switching event. Changing branch in
the opposite direction, the strongly shifted cavity returns to its initial
frequency, resulting in a drive far detuned of resonance. Thus, in both cases
switching leads to a frequency shift preventing to restore the initialised
state.

However, to further investigate the origin of this shift we measured
$\delta\mspace{-4.0mu}f$ for different temperatures as depicted in Fig. S.
5b). Keeping the input power constant at $n_{in}/n_{bi}=3.0$ we observe that
with increasing temperature the time scale as well as the total frequency
shift reduces. Furthermore, when the cavity is tuned to the right side of the
flux map instead of the left side, a positive frequency shift is observed
(Fig. S. 5c). Taken at the same bias frequency and with the same input power,
fitting the relaxation rate yields comparable parameters regardless on which
side it is measured. Contrary to the behaviour on the left fluxmap side, the
positive frequency shift on the right side does not allow to distinguish the
different branches, as shown in Fig. S. 5d). Irrespective of the branch which
is probed, the cavity reaches the same steady state frequency for a broad
range of initial frequencies. For high enough input powers $n_{in}/n_{bi}>1$,
a continuous switching between the branches can be observed as the shift
causes the drive to tune in and out of the resonance. With this behaviour
meaningful measurements beyond bifurcation on the right side of the fluxmap
are not possible. However, for input powers $n_{in}/n_{bi}<1$ the positive
shift acts as an effective negative feedbackloop stabilizing the resonance
against slow frequency fluctuations and thus proves beneficial for backaction
measurements before bistability, similar to observations made in [20].

Overall, the observed frequency shift scales with the intracavity photon
number. Amplitude and timescale decrease for higher temperatures and the
direction of the shift depends on the sign of the fluxmap slope. A frequency
shift due to radiation pressure on the mechanical resonator seems at first
glance plausible. Frequency tuning of our cavity by applying an external
magnetic field, leads to a circulating current within the SQUID due to flux
quantisation. The corresponding magnetic field of the SQUID displaces the
cantilever from its resting position. However, increasing the intracavity
photon number by driving the cavity decreases the circulating current in the
SQUID and the corresponding magnetic field. This in turn leads to a
displacement of the mechanical resonator towards its initial resting position.
Simultaneously, this changes the constant part of the magnetic flux from the
magnetic tip inside the SQUID resulting in a shifted resonance of the cavity.
Increasing the cavity population would lead to a lower circulating current,
which agrees with the observation, that the amplitude of the frequency shift
increases with input power and that the timescale is unaffected by it. Since
the linewidth of the mechanical resonator grows significantly with temperature
as shown in Fig. S. 4b), it seems plausible to assume, that this in turn
causes the temperature dependence of the timescale on which the shift
$\delta\mspace{-4.0mu}f$ is observed in Fig. S. 5b). Interestingly, the
relative changes in $\Gamma_{m}$ match with the changes in $\tau$, however the
absolute values are off by a factor of $\sim 2$. Nevertheless, for the
radiation pressure model the frequency shift should scale with
$\delta\mspace{-4.0mu}f\propto-g_{0}^{2}$, which contradicts the difference in
shift directions depending on the side of the fluxmap.

Another explanation could be that the Kerr-constant $\mathcal{K}$ of the
cavity is altered by the optomechanical coupling as stated in [10], reading

$\mathcal{K}_{\mathrm{eff}}=\mathcal{K}+\frac{2g_{0}^{2}\omega_{m}}{\omega_{m}^{2}+\frac{\Gamma_{m}^{2}}{4}}.$
(S. 4)

However, the change to $\mathcal{K}$ in the regime we operate the system at
would be too small to explain the observed shift and also fails to describe
the fluxmap side dependence due to its $g_{0}^{2}$ scaling.

Also magnetic surface spins are candidates to explain the properties of the
frequency shift. At low temperatures the spins would be aligned due to the
close proximity of the magnet at the cantilever tip. The ensemble of spins
creates a constant offset in magnetic flux at the SQUID loop. However,
strongly driving our cavity would cause some of the spins to scramble, which
would lead to a decrease in the offset magnetic flux at the SQUID. This would
result in a horizontal shift of the fluxmap towards absolute zero flux. Thus
the frequency on the left and right fluxmap side would shift in
$\delta\mspace{-4.0mu}f<0$ and $\delta\mspace{-4.0mu}f>0$, respectively,
matching the observations. Furthermore, larger input powers and therefore
higher AC-currents would create a large magnetic field change, which would
cause more spins to be scrambled. Therefore, the offset magnetic field would
decrease and thus shift the fluxmap by a larger extend. This would also match
with the obtained power dependence of the frequency shift. Lastly, the
relaxation time of the spins should feature the same timescale dependence on
temperature as observed for the frequency shift and, since higher temperatures
allow for a higher degree of initial disorder, it would also explain the
change in amplitude. Nevertheless, to quantify the influence on surface spins
experimentally proves to be challenging and further experimental advances have
to be made to verify this hypothesis.

##  Appendix S. 5 Cooling traces

In Fig. S. 6 additional cooling traces for input powers ranging from
$n_{in}/n_{bi}=0.5-3.0$ are shown, matching with the ones displayed in the
main text. The trace corresponding to the lowest input power is fitted with
the nonlinear theory taking the Kerr into account to extract the actual photon
number $n_{in}/n_{bi}=(0.54\pm 0.02)$ and the Kerr constant
$\mathcal{K}/2\pi=(14\pm 1)\,\mathrm{kHz}$ of the setup.

With those parameters the associated photon number is extrapolated for each
input power and the model is plotted to the experimental data. For each
individual power, the cooling trace is well described by the extrapolated
curves even though we increase the input power up to a factor of $6$ well
beyond the bifurcation point. As soon as we reach $n_{in}/n_{bi}>1$ two
distinct branches for the cooling are observed in good agreement with the
theory predictions. However, it is evident that except for the two sets where
we measured with the pulse tube turned off, we can not reach deep into the
cooling features. This indicates, that despite the vibration isolation
platform we still suffer from flux noise induced by mechanical vibrations in
the system. Nevertheless, we have to emphasise, that for being that far beyond
the bifurcation point the quality of the experimental data allows us to
clearly demonstrate, that backaction cooling and state manipulation in this
regime is possible and predictable.

Figure S. 6: Cooling traces at $g_{0}/2\pi=99\,\mathrm{Hz}$ with phonon number
$\langle n_{m}\rangle$, linewidth $\Gamma_{m}/2\pi$ and frequency shift
$\delta\omega_{m}/2\pi$ for various input powers $n_{in}/n_{bi}=0.5-3.0$. The
cooling trace at $n_{in}/n_{bi}=0.5$ is fitted to extract the system
parameters. For all other input powers the lines depict predictions from the
low power fit. Filled circles correspond to data taken with the pulsetube of
the cryostat turned off.