# Controlling the magnetic state of the proximate quantum spin liquid
$\alpha$-RuCl3 with an optical cavity

Emil Viñas Boström<EMAIL_ADDRESS>Max Planck Institute for the
Structure and Dynamics of Matter, Luruper Chaussee 149, 22761 Hamburg, Germany
Adithya Sriram  Department of Physics, Stanford University, Stanford,
California 94305, USA  Martin Claassen  Department of Physics and Astronomy,
University of Pennsylvania, Philadelphia, PA 19104  Center for Computational
Quantum Physics, Flatiron Institute, Simons Foundation, New York City, NY
10010, USA  Angel Rubio<EMAIL_ADDRESS>Max Planck Institute for the
Structure and Dynamics of Matter, Luruper Chaussee 149, 22761 Hamburg, Germany
Center for Computational Quantum Physics, Flatiron Institute, Simons
Foundation, New York City, NY 10010, USA

###### Abstract

Harnessing the enhanced light-matter coupling and quantum vacuum fluctuations
resulting from mode volume compression in optical cavities is a promising
route towards functionalizing quantum materials and realizing exotic states of
matter. Here, we extend cavity quantum electrodynamical materials engineering
to correlated magnetic systems, by demonstrating that a Fabry-Pérot cavity can
be used to control the magnetic state of the proximate quantum spin liquid
$\alpha$-RuCl3. Depending on specific cavity properties such as the mode
frequency, photon occupation, and strength of the light-matter coupling, any
of the magnetic phases supported by the extended Kitaev model can be
stabilized. In particular, in the THz regime, we show that the cavity vacuum
fluctuations alone are sufficient to bring $\alpha$-RuCl3 from a zigzag
antiferromagnetic to a ferromagnetic state. By external pumping of the cavity
in the few photon limit, it is further possible to push the system into the
antiferromagnetic Kitaev quantum spin liquid state.

## Introduction

The realization of magnetic van der Waals (vdW) materials with thicknesses
down to the monolayer limit has sparked a new interest in fundamental aspects
of two-dimensional magnetism. Gong17 ; Burch18 ; Gong19  Due to a competition
of strong anisotropy, fluctuations, and spin-orbit effects, vdW materials are
prime candidates to host exotic phenomena such as topological phase
transitions, magnetic skyrmions and quantum spin liquids. Savary2016 ;
Takagi19  In addition, the electronic, magnetic and optical properties of
these materials are sensitive to a wide range of material engineering
techniques such as strain, Kim2018 ; Cenker2022  nanostructuring, Moll2018
electric fields Huang2018 ; Jiang2018  and moiré twisting, Xie2021 ;
Kennes2021  allowing their state to be tuned with high precision.

Recent progress has also established optical engineering techniques as a
method to functionalize quantum materials and to reach exotic (out-of-
equilibrium) topological phases. McIver2019 ; Rudner2020 ; Shan2021 ;
Bloch2022  However, driving a system with lasers is associated with excessive
heating when the frequency becomes multi-photon resonant with electronic
transitions. DAlessio2014 ; Kennes2018  A way to circumvent this problem is to
embed the system in an optical cavity, where the effective light-matter
coupling is enhanced via mode volume compression and the state of the material
can be modified in an equilibrium setting. Ruggenthaler2018 ; Basov2020 ;
Hubener2020 ; Latini2021 ; Schlawin2022 ; Ashida2020 ; Curtis2022  Due to the
strong interaction between light and charged excitations polaritonic control
of material and chemical properties has, with a few exceptions, Sentef2020 ;
Chiocchetta2021 ; Zhang2022 ; Dirnberger2022  mainly been considered for
electronic states. Ebbesen2016 ; Sidler2021  While currently efforts are made
to extend the cavity framework to a broader class of materials, and to
construct a unified first principles description of cavity quantum
fluctuations and quantum matter, Flick2017 ; Ruggenthaler2018 ; Hubener2020
experiments demonstrating polaritonic control of materials are scarce.
Ebbesen2019 ; Fausti2022  Therefore, to transform this promising approach into
a powerful experimental tool, it is of key importance to identify candidate
materials where cavity engineering techniques can be explored.

Here, we extend the concept of cavity quantum electrodynamics (c-QED)
engineering into the magnetic regime and identify such a candidate system, by
demonstrating how an optical cavity can be used to control the magnetic ground
state of the proximate quantum spin liquid $\alpha$-RuCl3 via shaping the
quantum fluctuations of the cavity. Hubener2020 ; Ruggenthaler2018 ; Rokaj2019
; Sidler2021  Depending on the cavity frequency, photon occupation, and the
strength of the effective light-matter coupling, we find that it is possible
to transform the equilibrium zigzag antiferromagnetic order into any of the
magnetic phases supported by the extended Kitaev model (see Eq. 1 and Fig. 2).
As a key result we find that for frequencies of a few THz and for moderate
light-matter couplings, the interaction between the magnetic system and the
vacuum fluctuations of the cavity is sufficient to transform $\alpha$-RuCl3
from a zigzag antiferromagnet to a ferromagnet. In contrast to the meta-stable
states obtained by driving the system with classical light, the magnetic state
resulting from the interaction with the quantum fluctuations of the cavity is
a true equilibrium state denoted the photo ground state (PGS). Latini2021
Pumping the cavity in the few photon regime, it is further possible to push
the system into the Kitaev quantum spin liquid state and to retrieve the non-
equilibrium phase diagram of the semi-classical limit. Sriram2022  Our results
pave the way for utilizing c-QED to induce and control long-lived exotic
states in quantum materials.

Figure 1: Crystal and magnetic structure of monolayer $\alpha$-RuCl3. a,
Crystal structure of monolayer $\alpha$-RuCl3, with the magnetic Ru ions
(orange) arranged in a hexagonal lattice and exhibiting a zigzag
antiferromagnetic order. The surrounding octahedra of Cl ions give rise to a
crystal field splitting of the Ru $d$-orbitals (see panel c.) b, The magnetic
system interacts with a strength $g$ with a cavity electric field of frequency
$\Omega$. c, Energy level structure of the magnetic Ru ions, leading to an
effective $j_{\rm eff}=1/2$ magnetic moment in the Ru $t_{2g}$ manifold. d,
Magnetic interactions of Ru moments, colored according to the Kitaev bonds
$x$, $y$ and $z$.

## Results

### Low-energy model and equilibrium magnetic order

The vdW material $\alpha$-RuCl3 consists of layers of Ru atoms arranged in an
hexagonal lattice and surrounded by edge-sharing octahedra of Cl ions (Fig.
1a). Due to the crystal field the Ru $d$-orbitals are split by an energy
$\Delta_{\rm cf}$ into a lower $e_{g}$ and a higher $t_{2g}$ manifold, and the
strong spin-orbit coupling further splits the $t_{2g}$ states by an energy
$\Delta_{\rm soc}$ into a lower $j_{\rm eff}=3/2$ quartet, fully occupied in
the ground state, and a $j_{\rm eff}=1/2$ doublet with a single hole. The Cl
ions are assumed to be completely filled in the ground state, with their
$p$-orbitals separated from the Ru $d$-orbitals by a charge-transfer energy
$\Delta_{pd}$. The Ru energy level structure is schematically shown in Fig.
1c. The local interactions of the $t_{2g}$ manifold are described by a
Hubbard-Kanamori Hamiltonian $H_{U}$, Winter16 ; Sriram2022  which takes into
account the intra-orbital Hubbard interaction $U$, the inter-orbital
interaction $U^{\prime}$, the Hund’s coupling $J_{H}$ and the spin-orbit
coupling $\lambda$ (see Methods for a discussion of the model). The Ru holes
can either hop directly between Ru atoms or move via the Cl ligands, as
described by the kinetic Hamiltonian $H_{t}$. In the strong coupling limit
$t/U\ll 1$ virtual hopping processes give rise to effective magnetic
interactions via the superexchange mechanism. Due to the orbital alignment a
Kitaev interaction arises from ligand-mediated hopping over $90^{\circ}$ bond
angles, Jackeli09  while sub-dominant exchange and anisotropy terms arise from
direct Ru-Ru interactions.

Experimentally $\alpha$-RuCl3 is found to exhibit a zigzag antiferromagnetic
order below the Néel temperature $T_{N}\approx 7$ K, as indicated in Fig. 1.
At zero temperature, first principles calculations show that this zigzag state
is approximately degenerate with a ferromagnetic state, Kim2015  and might
only be stabilized by spin quantum fluctuations. Suzuki2021  In addition,
signatures of a Kitaev quantum spin liquid (QSL) state have been found upon
applying an external magnetic field along the out-of-plane direction.
Banerjee2016 ; Zheng2017  Together these results indicate that although
$\alpha$-RuCl3 orders at low temperatures, its ground state is proximate to
several competing magnetic orders and the magnetic phase diagram is determined
by a delicate competition of different magnetic interactions. This makes
$\alpha$-RuCl3 an excellent candidate material to explore the competition
between cavity and spin quantum fluctuations.

In the following the material will be assumed to have $C_{3}$ symmetry, which
is satisfied to a very good degree in $\alpha$-RuCl3. All parameters of the
local and kinetic Hamiltonians $H_{U}$ and $H_{t}$ were calculated from first
principles as discussed in the Methods section, and give a zigzag
antiferromagnetic ground state in line with observations.

### Light-matter coupling in a cavity

Within the low-energy description discussed above, the main effect of the
cavity is to modify the hopping amplitudes of the Ru holes. Inside the cavity
the total Hamiltonian is given by
$H=H_{U}+H_{t}+\sum_{\lambda}\hbar\Omega_{\lambda}\hat{n}_{\lambda}$, where
$\hbar\Omega_{\lambda}$ is the energy of photon mode $\lambda$ and
$\hat{n}_{\lambda}$ is the corresponding number operator. The photon field
modifies the kinetic Hamiltonian $H_{t}$ by introducing the replacements
$\hat{c}_{i\alpha\sigma}^{\dagger}\hat{c}_{i\beta\sigma}\to
e^{i\phi_{ij}}\hat{c}_{i\alpha\sigma}^{\dagger}\hat{c}_{i\beta\sigma}$, where
the Peierls phases $\phi_{ij}$ are proportional to the quantum vector
potential is $\hat{\bf A}=\sum_{\lambda}(g_{\lambda}{\bf
e}_{\lambda}\hat{a}_{\lambda}^{\dagger}+g_{\lambda}^{*}{\bf
e}^{*}_{\lambda}\hat{a}_{\lambda})$. For a given mode $\lambda$ the bare
light-matter coupling is defined as
$g_{\lambda}=ea/\sqrt{2\epsilon\hbar\Omega_{\lambda}V}$, where $e$ is the
elementary charge, $\hbar$ the reduced Planck constant, $\epsilon$ the
relative permittivity and $V$ the cavity mode volume. For a two-dimensional
Fabry-Pérot cavity the lowest energy photon mode has a frequency
$\Omega_{0}=\pi c/L_{z}$, and assuming a cavity with linear dimensions
$L_{x}=L_{y}=La$ where $L$ is width of the cavity in units of the Ru-Ru
distance $a$, the bare light-matter coupling of this mode is
$g_{0}=e/\sqrt{2\pi\epsilon\hbar cL^{2}}\approx 0.12/L$. However, below we
consider the effective single mode approximation obtained by integrating over
photon modes with momenta ${\bf q}\approx 0$, where the effective light-matter
coupling $g$ is determined by an effective mode volume $V_{\rm eff}$. Choi2017
; Hubener2020 ; Latini2021 ; Latini2022  Although the value of $V_{\rm eff}$
can in principle be fixed by comparisons to experiment, we here take it as a
free parameter.

Figure 2: Magnetic phases of the photo ground state. a, Spin cluster employed
for the exact diagonalization studies. b, Paths through the magnetic phase
diagram traced out by the system as the light-matter coupling $g$ increases
from $g_{\rm i}=0$ to $g_{\rm f}=0.5$. The paths are colored according to the
cavity photon energy $\hbar\Omega$, and the small arrows show the direction of
the paths. c, Parametrization angles $\theta$ and $\phi$ as a function of $g$
for $\hbar\Omega=10$ meV. d, Nearest neighbor spin-spin correlation functions
$\langle S_{i}^{z}S_{j}^{z}\rangle$. e, Expectation value of the plaquette
flux operator $W_{p}$. f, Average photon number $n_{\rm
av}=\langle\hat{n}\rangle$ of the photo ground state. g, Schematic of a fourth
order direct process showing a $1/\Omega$ enhancement at low photon energies.
h, Schematic of a fourth order ligand mediated process unaffected by the low-
frequency limit.

### Effective coupled spin-photon model

The total Hamiltonian is down-folded to the spin sector by eliminating $H_{t}$
to fourth order in virtual ligand mediated processes using quasi-degenerate
perturbation theory. The structure of the perturbation expansion allows for
the down-folding to be performed separately within each given photon sector,
resulting in a coupled spin-photon Hamiltonian of the form

$\displaystyle\mathcal{H}$ $\displaystyle=\sum_{\bf
nm}\big{(}\mathcal{H}_{s,\bf nm}+\delta_{\bf
nm}\sum_{\lambda}\hbar\Omega_{\lambda}n_{\lambda}\big{)}|{\bf
n}\rangle\langle{\bf m}|.$ (1)

Here $\mathcal{H}_{s,\bf nm}$ is the spin Hamiltonian in the sector connecting
cavity number states $|\bf n\rangle$ and $|\bf m\rangle$, where ${\bf
m}=\\{m_{1},m_{2},\ldots,m_{L}\\}$ for $L$ modes. The Hamiltonian in each
photon sector is

$\displaystyle\mathcal{H}_{s,\bf nm}$ $\displaystyle=\sum_{\langle
ij\rangle}\begin{pmatrix}S_{i}^{\alpha}&S_{i}^{\beta}&S_{i}^{\gamma}\end{pmatrix}\begin{pmatrix}J&\Gamma&\Gamma^{\prime}\\\
\Gamma&J&\Gamma^{\prime}\\\
\Gamma^{\prime}&\Gamma^{\prime}&J+K\end{pmatrix}_{\bf
nm}\begin{pmatrix}S_{j}^{\alpha}\\\ S_{j}^{\beta}\\\
S_{j}^{\gamma}\end{pmatrix}$ $\displaystyle+B_{\bf nm}\sum_{i}\hat{\bf
e}_{B}\cdot{\bf S}_{i}$ (2)

where the magnetic interactions $J$, $K$, $\Gamma$ and $\Gamma^{\prime}$ all
depend on the light-matter coupling ${\bf g}=\\{g_{1},g_{2},\ldots,g_{L}\\}$
and the photon numbers ${\bf n}$ and ${\bf m}$. Each bond $\langle ij\rangle$
is labeled by the indexes $\alpha\beta(\gamma)\in\\{xy(z),yz(x),zx(y)\\}$ and
is denoted a $\gamma$-bond in accordance with Fig. 1d. The induced magnetic
field $B$ points along the $[111]$ direction of the local spin axes, as
defined by the unit vector $\hat{\bf e}_{B}$.

The form of the spin-photon Hamiltonian in Eq. Effective coupled spin-photon
model is valid as long as the $C_{3}$ symmetry is preserved. Since a single
linearly polarized mode breaks rotational symmetry, such a mode induces
additional terms in the spin Hamiltonian. The rotational symmetry breaking is
also reflected in a strong dependence of the cavity-modified spin parameters
on the polarization direction, as shown in Supplemental Figure S1. However, in
a Fabry-Pérot cavity the lowest mode is doubly degenerate with two possible
in-plane polarizations, and including both these modes in the description
restores the rotational symmetry of $\mathcal{H}$. The $C_{3}$ symmetry can
also be maintained by considering a single circularly polarized mode in a
chiral cavity, Hubener2020  as will be done in the following.

Figure 3: Magnetic phases in a seeded cavity. a, Paths through the magnetic
phase diagram traced out by the spin system as the light-matter coupling $g$
increases from $g_{\rm i}=0$ to $g_{\rm f}=0.5$. The paths are colored
according to the cavity photon energy $\hbar\Omega$, and the small arrows show
the direction of the paths. b, Parametrization angles $\theta$ and $\phi$ as a
function of $g$ for $\hbar\Omega=3$ eV. c, Spin-spin correlation function
$\langle S_{i}^{z}S_{j}^{z}\rangle$ as a function of photon energy and light-
matter coupling. d, Average of the plaquette flux operator $W_{p}$ as a
function of photon energy and light-matter coupling. e, Schematic of third
order processes where hopping around isosceles triangles leads to an induced
magnetic field. In all panels the average photon number is $n_{\rm av}=1$.

### Magnetic phases of the cavity photo ground state

The PGS and magnetic phase diagram of the cavity-embedded spin system was
obtained by exact diagonalization of Eq. 1 on the 24-site spin cluster shown
in Fig. 2, including a single effective circularly polarized cavity mode. This
spin cluster is the minimal one known to respect all sub-lattice symmetries of
the magnetic system. Chaloupka2015  To perform the calculations we assume a
perfect cavity, while a more quantitative description should account for
dissipation. However, the results discussed below are expected to be robust
against such effects.

To find how the magnetic ground state flows through the phase diagram of the
extended Kitaev model as a function of light-matter coupling and cavity
frequency (see Fig. 2b), the magnetic interactions are parameterized by
$\bar{J}=\sin\theta\cos\phi$, $\bar{K}=\sin\theta\sin\phi$ and
$\bar{\Gamma}=\cos\theta$. Here $\bar{J}$, $\bar{K}$ and $\bar{\Gamma}$ are
the cavity-renormalized spin parameters divided by the energy
$E=\sqrt{J^{2}+K^{2}+\Gamma^{2}}$. The spin parameters for $g=0$ were obtained
via first principles calculations as discussed in the Methods section, and
give $\phi=-89.43^{\circ}$ and $\theta=69.44^{\circ}$ placing the equilibrium
system in a zigzag antiferromagnetic state adjacent to the ferromagnetic
Kitaev QSL. This is in good agreement with experimental findings. Banerjee2016
; Zheng2017  To identify the magnetic state we calculate the spin-spin
correlation function $\langle S_{i}^{z}S_{j}^{z}\rangle$ and the expectation
value of the plaquette flux operator
$W_{p}=2^{-6}S_{i}^{x}S_{j}^{y}S_{k}^{z}S_{l}^{x}S_{m}^{y}S_{n}^{z}$. In
$W_{p}$ the site indexes traverses a given hexagon $p$ in the positive
direction, and the spin component at a given site is determined by the bond
pointing out from the hexagon. Magnetic phase transitions are identified by
the second order derivatives of the total energy.

Fig. 2b shows the paths traced out by the magnetic ground state as $g$ is
increased from the initial value $g_{i}=0$ to the final value $g_{f}=0.5$, for
a number of photon energies in the range $\hbar\Omega\in[0,0.2]$ eV. For all
parameters the system flows away from the ferromagnetic Kitaev point, and
depending on $\hbar\Omega$ either remain in the zigzag state or enter the
ferromagnetic domain. In particular, for photon energies below
$\hbar\Omega\approx 10$ meV, the PGS evolves from a zigzag antiferromagnetic
state into a ferromagnetic state as the light-matter coupling is increased
(see Figs. 2d and 2e). We remark that light-matter couplings of this magnitude
have been achieved in experiments on two-dimensional materials in driven
cavities. Liu2014 ; Li2018 ; ParaviciniBagliani2018  The increase of $g$ is
associated with an increase in photon occupation, which diverges in the limit
$\Omega\to 0$ (see Fig. 2f). The reason for this divergence is a class of
fourth order virtual electronic processes where a single charge excitation
moves while simultaneously emitting/absorbing a virtual cavity photon (see
Fig. 2g). Since the magnetic interaction parameters are inversely proportional
to the energy difference between connected virtual states, this leads to a
$J\sim 1/\Omega$ enhancement in the low frequency limit. This behavior has
been validated in small electron-photon clusters (see Supplementary Figure
S2), which show good qualitative agreement with the larger-scale spin-photon
model simulations.

The microscopic origin of the magnetic phase transition is a simultaneous
suppression of the Kitaev interaction and increase of the isotropic exchange,
as seen from the normalized spin parameters of the zero photon sector
displayed in Supplementary Figure S3. In particular, the Kitaev interaction
changes sign as a function of light-matter coupling at a point coinciding with
the magnetic phase transition. This change in parameters is consistent with a
simultaneous suppression of the effective ligand mediated hopping and increase
of direct Ru$-$Ru hopping. To see how this comes about we note that while the
direct hopping process depicted in Fig. 2g implies an enhancement of $J\sim
1/\Omega$ at small frequencies, the ligand mediated process shown in Fig. 2h
is proportional to the reciprocal of the charge-transfer energy. Therefore
$K\sim 1/\Delta_{pd}$ is largely unaffected by the low frequency limit,
leading to a relative enhancement of $J/K$.

### Magnetic phase diagram of a pumped cavity

The results of Fig. 2 show that the quantum fluctuations of the cavity vacuum
are sufficient to bring about a change in the magnetic ground state of
$\alpha$-RuCl3 in the THz regime. In contrast, for photon energies large
compared to the magnetic energy scale ($K/\hbar\Omega\ll 1$), the fluctuations
of real photons are strongly suppressed. This restricts the system to a single
photon sector, and the Hamiltonian is well approximated by keeping only the
diagonal terms $\mathcal{H}_{s,\bf nm}=\mathcal{H}_{s,\bf nn}\delta_{\bf nm}$.
In this regime the spin and photon fluctuations therefore decouples, and the
magnetic ground state can be obtained from a pure spin model with effective
cavity-renormalized parameters.

The magnetic phase diagram of $\alpha$-RuCl3, interacting with a single
effective circularly polarized cavity mode with average photon occupation
$n_{\rm av}=1$, is shown in Fig. 3. The results are displayed as a function of
the effective light-matter coupling $\bar{g}=2g\sqrt{n_{\rm av}}$, and again
the cavity is seen to make the magnetic ground state flow towards the
antiferromagnetic Kitaev point (Fig. 3a). The cavity has a drastic effect on
the spin parameters as the photon energy is tuned through the charge
resonances of the underlying electronic system, as illustrated by the sharp
variations of the spin-spin correlation function and plaquette flux around
$\hbar\Omega=2$ eV (see Figs. 3c and 3d as well as Supplementary Figure S4).
For a given light-matter coupling this leads to a series of magnetic phase
transitions as a function of frequency, stabilizing quasi-stationary states
with either ferromagnetic, antiferromagnetic or spiral order depending on the
parameters.

In addition, there is a large region of light-matter couplings
$\bar{g}=0.2-0.6$ and photon energies $\hbar\Omega=2.8-3.2$ eV where the
system enters the antiferromagnetic Kitaev QSL state. The presence of a Kitaev
QSL has been verified by calculations of the spin-spin correlation function as
well as the plaquette flux operator $W_{p}$, which takes on a quantized value
$W_{p}=\pm 1$ in the QSL phase. Takagi19  Although the equilibrium system
displays a ferromagnetic Kitaev interaction, the QSL state appears in a region
of antiferromagnetic Kitaev interaction implying that the cavity induces a
sign reversal of $K$. The QSL is found to be stabilized by the effective
magnetic field induced by the cavity (Fig. 3e), as well as a relative increase
of the $\Gamma^{\prime}$ interaction.

As seen from Fig. 3, $\alpha$-RuCl3 also enters the QSL phase in a narrow
region of cavity mode frequencies around $\hbar\Omega=2.1$ eV. This coincides
with a region where the photo-modified exchange interaction $J$ changes sign,
and therefore is very small. More specifically, since the photo-modified
magnetic exchange is given close to a resonance by $J\sim 1/(\Omega-E_{\rm
res})$, and two resonances appear close together at about $\hbar\Omega=2.0$ eV
and $\hbar\Omega=2.2$ eV, the different sign of $J$ above the lower and below
the upper resonance implies a sign reversal between the resonances. This gives
a relative amplification of the Kitaev interaction $K$ inducing the QSL state.

## Discussion

The results of Fig. 2 and 3 show that the magnetic ground state of
$\alpha$-RuCl3 is sensitive to the interaction with the quantum fluctuations
of the cavity field, and depending on the photon frequency, average cavity
occupation and strength of the light-matter coupling, the equilibrium magnetic
order can be transformed into any of the magnetic states supported by the
extended Kitaev model of Eq. 1. In particular, our results show that the
cavity vacuum fluctuations alone are indeed sufficient to change the magnetic
order of the photo ground state when the light-matter coupling is sufficiently
strong (but within the range of present experimental light-matter couplings).
However, as the value of the light-matter coupling is strongly dependent on
the cavity size, and scales as $g\sim 1/L$ with the linear size of the system,
it seems as though the effect of the cavity vanishes quickly in the
thermodynamic limit. This naive scaling argument might fail for several
reasons, the most important being the neglect of additional cavity modes with
momenta ${\bf q}\approx 0$. In fact, the number of modes with momenta smaller
than some given cut-off $q_{\rm max}$ scales as $N(q_{\rm max})\sim L$,
Latini2021 ; Latini2021b ; Fausti2022  and can potentially cancel the volume
scaling of $g$. For the effective single mode approximation employed here, the
mode volume appearing in the light-matter coupling should therefore be
interpreted as an effective mode volume to be fixed either by additional
calculations or by comparison to experiment. Choi2017 ; Hubener2020 ;
Latini2021

An additional subtlety of the thermodynamic limit is the potential coupling of
the cavity modes to collective fluctuations of the magnetic system. In
particular, close to a magnetic phase boundary, collective fluctuations with a
correlation length $\xi\sim L$ are expected to appear. Such collective modes
can be expected to play a crucial role in stabilizing the magnetic order, and
to couple more strongly to the cavity field. Weber2022  However, an explicit
treatment of such macroscopic effects, while simultaneously retaining an exact
description of spin and photon quantum fluctuations, requires a more advanced
methodology going beyond the current work.

For practical purposes, we note that for a given light-matter coupling $g$ the
effective coupling $\bar{g}$ can be enhanced by increasing the photon
occupation of the cavity. In particular, since $\bar{g}=2g\sqrt{n_{\rm av}}$,
a substantial effective coupling (and corresponding modification of the
magnetic interactions in the spin-photon Hamiltonian of Eq. 1) can be obtained
with relatively few photons. Although the present work neglects any cavity
dissipation as might be relevant in a non-equilibrium setting or for a dark
cavity with lossy mirrors, the qualitative conclusions are expected to be
robust against such effects. This includes in particular the identification of
a zigzag antiferromagnetic to ferromagnetic transition induced by the cavity
vacuum fluctuations, and the presence of a large Kitaev QSL domain in the
magnetic phase diagram in the few photon regime. Together these results
demonstrate the feasibility of c-QED engineering of the magnetic state of the
proximate spin liquid $\alpha$-RuCl3 in the few photon limit. Looking ahead,
we note that the cavity photon momentum constitutes a tuning parameter in
addition to the light-matter coupling, and we therefore envisage that c-QED
can be used to also modify phases with finite-${\bf q}$ ordering vectors (such
as charge density waves). Our work thereby paves the way for utilizing c-QED
to control exotic magnetic states in real cavity-embedded quantum materials.
Ruggenthaler2018 ; Hubener2020

## References

  * (1) Gong, C. _et al._ Discovery of intrinsic ferromagnetism in two-dimensional van der waals crystals.  _Nature_ 546, 265–269 (2017).  URL https://doi.org/10.1038/nature22060.
  * (2) Burch, K. S., Mandrus, D. & Park, J.-G.  Magnetism in two-dimensional van der Waals materials.  _Nature_ 563, 47–52 (2018).  URL https://doi.org/10.1038/s41586-018-0631-z.
  * (3) Gong, C. & Zhang, X.  Two-dimensional magnetic crystals and emergent heterostructure devices.  _Science_ 363, eaav4450 (2019).  URL https://doi.org/10.1126/science.aav4450.
  * (4) Savary, L. & Balents, L.  Quantum spin liquids: a review.  _Reports on Progress in Physics_ 80, 016502 (2016).  URL https://doi.org/10.1088/0034-4885/80/1/016502.
  * (5) Takagi, H., Takayama, T., Jackeli, G., Khaliullin, G. & Nagler, S. E.  Concept and realization of Kitaev quantum spin liquids.  _Nature Reviews Physics_ 1, 264–280 (2019).  URL https://doi.org/10.1038/s42254-019-0038-2.
  * (6) Kim, H.-H. _et al._ Uniaxial pressure control of competing orders in a high-temperature superconductor.  _Science_ 362, 1040–1044 (2018).  URL https://doi.org/10.1126/science.aat4708.
  * (7) Cenker, J. _et al._ Reversible strain-induced magnetic phase transition in a van der Waals magnet.  _Nature Nanotechnology_ 17, 256–261 (2022).  URL https://doi.org/10.1038/s41565-021-01052-6.
  * (8) Moll, P. J.  Focused ion beam microstructuring of quantum matter.  _Annual Review of Condensed Matter Physics_ 9, 147–162 (2018).  URL https://doi.org/10.1146/annurev-conmatphys-033117-054021.
  * (9) Huang, B. _et al._ Electrical control of 2D magnetism in bilayer CrI3.  _Nature Nanotechnology_ 13, 544–548 (2018).  URL https://doi.org/10.1038/s41565-018-0121-3.
  * (10) Jiang, S., Shan, J. & Mak, K. F.  Electric-field switching of two-dimensional van der Waals magnets.  _Nature Materials_ 17, 406–410 (2018).  URL https://doi.org/10.1038/s41563-018-0040-6.
  * (11) Xie, H. _et al._ Twist engineering of the two-dimensional magnetism in double bilayer chromium triiodide homostructures.  _Nature Physics_ 18, 30–36 (2021).  URL https://doi.org/10.1038/s41567-021-01408-8.
  * (12) Kennes, D. M. _et al._ Moiré heterostructures as a condensed-matter quantum simulator.  _Nature Physics_ 17, 155–163 (2021).  URL https://doi.org/10.1038/s41567-020-01154-3.
  * (13) McIver, J. W. _et al._ Light-induced anomalous Hall effect in graphene.  _Nature Physics_ 16, 38–41 (2019).  URL https://doi.org/10.1038/s41567-019-0698-y.
  * (14) Rudner, M. S. & Lindner, N. H.  Band structure engineering and non-equilibrium dynamics in Floquet topological insulators.  _Nature Reviews Physics_ 2, 229–244 (2020).  URL https://doi.org/10.1038/s42254-020-0170-z.
  * (15) Shan, J.-Y. _et al._ Giant modulation of optical nonlinearity by Floquet engineering.  _Nature_ 600, 235–239 (2021).  URL https://doi.org/10.1038/s41586-021-04051-8.
  * (16) Bloch, J., Cavalleri, A., Galitski, V., Hafezi, M. & Rubio, A.  Strongly correlated electron–photon systems.  _Nature_ 606, 41–48 (2022).  URL https://doi.org/10.1038/s41586-022-04726-w.
  * (17) D’Alessio, L. & Rigol, M.  Long-time behavior of isolated periodically driven interacting lattice systems.  _Phys. Rev. X_ 4, 041048 (2014).  URL https://link.aps.org/doi/10.1103/PhysRevX.4.041048.
  * (18) Kennes, D. M., de la Torre, A., Ron, A., Hsieh, D. & Millis, A. J.  Floquet engineering in quantum chains.  _Phys. Rev. Lett._ 120, 127601 (2018).  URL https://link.aps.org/doi/10.1103/PhysRevLett.120.127601.
  * (19) Ruggenthaler, M., Tancogne-Dejean, N., Flick, J., Appel, H. & Rubio, A.  From a quantum-electrodynamical light–matter description to novel spectroscopies.  _Nature Reviews Chemistry_ 2 (2018).  URL https://doi.org/10.1038/s41570-018-0118.
  * (20) Basov, D. N., Asenjo-Garcia, A., Schuck, P. J., Zhu, X. & Rubio, A.  Polariton panorama.  _Nanophotonics_ 10, 549–577 (2020).  URL https://doi.org/10.1515/nanoph-2020-0449.
  * (21) Hübener, H. _et al._ Engineering quantum materials with chiral optical cavities.  _Nature Materials_ 20, 438–442 (2020).  URL https://doi.org/10.1038/s41563-020-00801-7.
  * (22) Latini, S. _et al._ The ferroelectric photo ground state of SrTiO3: Cavity materials engineering.  _Proceedings of the National Academy of Sciences_ 118 (2021).  URL https://doi.org/10.1073/pnas.2105618118.
  * (23) Schlawin, F., Kennes, D. M. & Sentef, M. A.  Cavity quantum materials.  _Applied Physics Reviews_ 9, 011312 (2022).  URL https://doi.org/10.1063/5.0083825.
  * (24) Ashida, Y. _et al._ Quantum electrodynamic control of matter: Cavity-enhanced ferroelectric phase transition.  _Phys. Rev. X_ 10, 041027 (2020).  URL https://link.aps.org/doi/10.1103/PhysRevX.10.041027.
  * (25) Curtis, J. B. _et al._ Cavity magnon-polaritons in cuprate parent compounds.  _Phys. Rev. Research_ 4, 013101 (2022).  URL https://link.aps.org/doi/10.1103/PhysRevResearch.4.013101.
  * (26) Sentef, M. A., Li, J., Künzel, F. & Eckstein, M.  Quantum to classical crossover of Floquet engineering in correlated quantum systems.  _Physical Review Research_ 2 (2020).  URL https://doi.org/10.1103/physrevresearch.2.033033.
  * (27) Chiocchetta, A., Kiese, D., Zelle, C. P., Piazza, F. & Diehl, S.  Cavity-induced quantum spin liquids.  _Nature Communications_ 12 (2021).  URL https://doi.org/10.1038/s41467-021-26076-3.
  * (28) Zhang, H. _et al._ Cavity-enhanced linear dichroism in a van der Waals antiferromagnet.  _Nature Photonics_ 16, 311–317 (2022).  URL https://doi.org/10.1038/s41566-022-00970-8.
  * (29) Dirnberger, F. _et al._ Spin-correlated exciton–polaritons in a van der Waals magnet.  _Nature Nanotechnology_ (2022).  URL https://doi.org/10.1038/s41565-022-01204-2.
  * (30) Ebbesen, T. W.  Hybrid light–matter states in a molecular and material science perspective.  _Accounts of Chemical Research_ 49, 2403–2412 (2016).  URL https://doi.org/10.1021/acs.accounts.6b00295.
  * (31) Sidler, D., Ruggenthaler, M., Schäfer, C., Ronca, E. & Rubio, A.  A perspective on ab initio modeling of polaritonic chemistry: The role of non-equilibrium effects and quantum collectivity (2021).  eprint arXiv:2108.12244.
  * (32) Flick, J., Ruggenthaler, M., Appel, H. & Rubio, A.  Atoms and molecules in cavities, from weak to strong coupling in quantum-electrodynamics (QED) chemistry.  _Proceedings of the National Academy of Sciences_ 114, 3026–3034 (2017).  URL https://doi.org/10.1073/pnas.1615509114.
  * (33) Thomas, A. _et al._ Exploring superconductivity under strong coupling with the vacuum electromagnetic field (2019).  eprint arXiv:1911.01459.
  * (34) Jarc, G. _et al._ Cavity control of the metal-to-insulator transition in 1T-TaS2 (2022).  eprint arXiv:2210.02346.
  * (35) Rokaj, V., Penz, M., Sentef, M. A., Ruggenthaler, M. & Rubio, A.  Quantum electrodynamical Bloch theory with homogeneous magnetic fields.  _Phys. Rev. Lett._ 123, 047202 (2019).  URL https://link.aps.org/doi/10.1103/PhysRevLett.123.047202.
  * (36) Sriram, A. & Claassen, M.  Light-induced control of magnetic phases in Kitaev quantum magnets.  _Phys. Rev. Research_ 4, L032036 (2022).  URL https://link.aps.org/doi/10.1103/PhysRevResearch.4.L032036.
  * (37) Winter, S. M., Li, Y., Jeschke, H. O. & Valentí, R.  Challenges in design of Kitaev materials: Magnetic interactions from competing energy scales.  _Phys. Rev. B_ 93, 214431 (2016).  URL https://link.aps.org/doi/10.1103/PhysRevB.93.214431.
  * (38) Jackeli, G. & Khaliullin, G.  Mott insulators in the strong spin-orbit coupling limit: From Heisenberg to a quantum compass and Kitaev models.  _Physical Review Letters_ 102 (2009).  URL https://doi.org/10.1103/physrevlett.102.017205.
  * (39) Kim, H.-S., V., V. S., Catuneanu, A. & Kee, H.-Y.  Kitaev magnetism in honeycomb RuCl3 with intermediate spin-orbit coupling.  _Phys. Rev. B_ 91, 241110 (2015).  URL https://link.aps.org/doi/10.1103/PhysRevB.91.241110.
  * (40) Suzuki, H. _et al._ Proximate ferromagnetic state in the Kitaev model material $\alpha-$RuCl3.  _Nature Communications_ 12 (2021).  URL https://doi.org/10.1038%2Fs41467-021-24722-4.
  * (41) Banerjee, A. _et al._ Proximate Kitaev quantum spin liquid behaviour in a honeycomb magnet.  _Nature Materials_ 15, 733–740 (2016).  URL https://doi.org/10.1038/nmat4604.
  * (42) Zheng, J. _et al._ Gapless spin excitations in the field-induced quantum spin liquid phase of $\alpha-$RuCl3.  _Phys. Rev. Lett._ 119, 227208 (2017).  URL https://link.aps.org/doi/10.1103/PhysRevLett.119.227208.
  * (43) Choi, H., Heuck, M. & Englund, D.  Self-similar nanocavity design with ultrasmall mode volume for single-photon nonlinearities.  _Phys. Rev. Lett._ 118, 223605 (2017).  URL https://link.aps.org/doi/10.1103/PhysRevLett.118.223605.
  * (44) Latini, S., Hübener, H., Ruggenthaler, M. & Rubio, A.  QED light-matter coupling: Macroscopic limit in the dipole approximation (2022).  eprint In preparation.
  * (45) Chaloupka, J. & Khaliullin, G.  Hidden symmetries of the extended kitaev-heisenberg model: Implications for the honeycomb-lattice iridates A2IrO3.  _Phys. Rev. B_ 92, 024413 (2015).  URL https://link.aps.org/doi/10.1103/PhysRevB.92.024413.
  * (46) Liu, X. _et al._ Strong light–matter coupling in two-dimensional atomic crystals.  _Nature Photonics_ 9, 30–34 (2014).  URL https://doi.org/10.1038/nphoton.2014.304.
  * (47) Li, X. _et al._ Observation of Dicke cooperativity in magnetic interactions.  _Science_ 361, 794–797 (2018).  URL https://doi.org/10.1126/science.aat5162.
  * (48) Paravicini-Bagliani, G. L. _et al._ Magneto-transport controlled by Landau polariton states.  _Nature Physics_ 15, 186–190 (2018).  URL https://doi.org/10.1038/s41567-018-0346-y.
  * (49) Latini, S. _et al._ Phonoritons as hybridized exciton-photon-phonon excitations in a monolayer $h$-BN optical cavity.  _Phys. Rev. Lett._ 126, 227401 (2021).  URL https://link.aps.org/doi/10.1103/PhysRevLett.126.227401.
  * (50) Weber, L., Boström, E. V., Claassen, M., Rubio, A. & Kennes, D. M.  Cavity-renormalized quantum criticality in a honeycomb bilayer antiferromagnet (2022).  eprint In preparation.
  * (51) Li, J. & Eckstein, M.  Manipulating intertwined orders in solids with quantum light.  _Phys. Rev. Lett._ 125, 217402 (2020).  URL https://link.aps.org/doi/10.1103/PhysRevLett.125.217402.
  * (52) Li, J., Schamriß, L. & Eckstein, M.  Effective theory of lattice electrons strongly coupled to quantum electromagnetic fields.  _Phys. Rev. B_ 105, 165121 (2022).  URL https://link.aps.org/doi/10.1103/PhysRevB.105.165121.
  * (53) Winkler, R.  _Quasi-Degenerate Perturbation Theory_ , 201–205 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2003).  URL https://doi.org/10.1007/978-3-540-36616-4_12.
  * (54) Weinberg, P. & Bukov, M.  QuSpin: a Python package for dynamics and exact diagonalisation of quantum many body systems. Part I: spin chains.  _SciPost Phys._ 2, 003 (2017).  URL https://scipost.org/10.21468/SciPostPhys.2.1.003.
  * (55) Weinberg, P. & Bukov, M.  QuSpin: a Python package for dynamics and exact diagonalisation of quantum many body systems. Part II: bosons, fermions and higher spins.  _SciPost Phys._ 7, 020 (2019).  URL https://scipost.org/10.21468/SciPostPhys.7.2.020.
  * (56) Tancogne-Dejean, N., Oliveira, M. J. T. & Rubio, A.  Self-consistent DFT$+{U}$ method for real-space time-dependent density functional theory calculations.  _Phys. Rev. B_ 96, 245133 (2017).  URL https://link.aps.org/doi/10.1103/PhysRevB.96.245133.
  * (57) Tancogne-Dejean, N. _et al._ Octopus, a computational framework for exploring light-driven phenomena and quantum dynamics in extended and finite systems.  _The Journal of Chemical Physics_ 152, 124119 (2020).  URL https://doi.org/10.1063/1.5142502.
  * (58) Marzari, N. & Vanderbilt, D.  Maximally localized generalized Wannier functions for composite energy bands.  _Phys. Rev. B_ 56, 12847–12865 (1997).  URL https://link.aps.org/doi/10.1103/PhysRevB.56.12847.
  * (59) Souza, I., Marzari, N. & Vanderbilt, D.  Maximally localized Wannier functions for entangled energy bands.  _Phys. Rev. B_ 65, 035109 (2001).  URL https://link.aps.org/doi/10.1103/PhysRevB.65.035109.

## Methods

### Effective low-energy lattice electron Hamiltonian

The crystal structure of a single layer $\alpha$-RuCl3 consists of a hexagonal
lattice of Ru atoms, surrounded by edge-sharing octahedra of Cl atoms. Due to
the strong octahedral crystal field the Ru3+ $d$-orbitals are split into a
lower $e_{g}$ and a higher $t_{2g}$ manifold, the latter consisting of the
orbitals $d_{xy}$, $d_{xz}$ and $d_{yz}$. Due to the strong spin-orbit
coupling the $t_{2g}$ states are further split into a lower $j_{\rm eff}=3/2$
quartet, fully occupied in the ground state, and a $j_{\rm eff}=1/2$ doublet
with a single hole. The Cl- ions are assumed to be completely filled in the
ground state, with their $p_{x}$, $p_{y}$ and $p_{z}$ orbitals separated from
the $d$-orbitals by a charge-transfer energy $\Delta_{pd}$.

The local part of the Hamiltonian is given by Winter16 ; Sriram2022

$\displaystyle H_{U}$
$\displaystyle=U\sum_{i\alpha}\hat{n}_{i\alpha\uparrow}\hat{n}_{i\alpha\downarrow}+\sum_{i\sigma\sigma^{\prime},\alpha<\beta}(U^{\prime}-J_{H}\delta_{\sigma\sigma^{\prime}})\hat{n}_{i\alpha\sigma}\hat{n}_{i\beta\sigma^{\prime}}$
$\displaystyle+J_{H}\sum_{i,\alpha\neq\beta}(\hat{c}_{i\alpha\uparrow}^{\dagger}\hat{c}_{i\alpha\downarrow}^{\dagger}\hat{c}_{i\beta\downarrow}\hat{c}_{i\beta\uparrow}-\hat{c}_{i\alpha\uparrow}^{\dagger}\hat{c}_{i\alpha\downarrow}\hat{c}_{i\beta\downarrow}^{\dagger}\hat{c}_{i\beta\uparrow})$
$\displaystyle+\Delta_{pd}\sum_{i^{\prime}\sigma}\hat{n}_{i\sigma}^{p}+\frac{\lambda}{2}\sum_{i}\hat{\bf
c}_{i}^{\dagger}({\bf L}\cdot{\bf s})\hat{\bf c}_{i}.$ (3)

Here $U$ is the intraorbital Hubbard interaction, $U^{\prime}$ the
interorbital interaction and $J_{H}$ the Hund’s coupling between the Ru
$d$-orbitals $\alpha,\beta\in\\{yz,xz,xy\\}$. Further, $\Delta_{pd}$ is the
single-particle charge transfer energy to add a hole to the Cl ligands,
$\lambda$ is the strength of the spin-orbit coupling (SOC), and the vector of
operators is $\hat{\bf
c}_{i}^{\dagger}=(\hat{c}_{iyz\uparrow}^{\dagger},\hat{c}_{iyz\downarrow}^{\dagger},\hat{c}_{ixz\uparrow}^{\dagger},\hat{c}_{ixz\downarrow}^{\dagger},\hat{c}_{ixy\uparrow}^{\dagger},\hat{c}_{ixy\downarrow}^{\dagger})$.
In this basis, the inner product of orbital and spin angular momentum may be
written

$\displaystyle{\bf L}\cdot{\bf s}=\begin{pmatrix}0&-i\sigma_{z}&i\sigma_{y}\\\
i\sigma_{z}&0&-i\sigma_{x}\\\ -i\sigma_{y}&i\sigma_{x}&0\end{pmatrix}.$ (4)

To account for hopping processes between the Ru $d$-orbitals and Cl
$p$-orbitals, we assume the system has $C_{3}$ symmetry. The hopping processes
along a $Z$-bond are described by

$\displaystyle H_{t}^{\prime}$ $\displaystyle=\sum_{\langle
ij\rangle\sigma}\bigg{[}\Big{(}\hat{c}_{iyz\sigma}^{\dagger}\,\hat{c}_{ixz\sigma}^{\dagger}\,\hat{c}_{ixy\sigma}^{\dagger}\Big{)}\begin{pmatrix}t_{1}&t_{2}&t_{4}\\\
t_{2}&t_{1}&t_{4}\\\
t_{4}&t_{4}&t_{3}\end{pmatrix}\begin{pmatrix}\hat{c}_{iyz\sigma}\\\
\hat{c}_{ixz\sigma}\\\ \hat{c}_{ixy\sigma}\end{pmatrix}$
$\displaystyle+t_{pd}\big{(}\hat{p}_{i^{\prime}\sigma}^{\dagger}\hat{c}_{ixz\sigma}+\hat{c}_{jyz\sigma}^{\dagger}\hat{p}_{i^{\prime}\sigma}$
$\displaystyle\hskip
15.6491pt+\hat{p}_{j^{\prime}\sigma}^{\dagger}\hat{c}_{iyz\sigma}+\hat{c}_{jxz\sigma}^{\dagger}\hat{p}_{j^{\prime}\sigma}\big{)}+H.c.\bigg{]}.$
(5)

Here, $t_{i}$ with $i\in\\{1,2,3,4\\}$ parameterize direct $d-d$ hopping
processes, while $t_{pd}$ determines the strength of ligand mediated hopping
via the Cl $p_{z}$-orbitals.

### Light-matter coupling

In presence of the cavity the total Hamiltonian in the dipole approximation is
given by $H=H_{U}+H_{t}+\sum_{\lambda}\hbar\Omega_{\lambda}\hat{n}_{\lambda}$.
Here, $\hbar\Omega_{\lambda}$ is the photon energy of a mode described by the
label $\lambda$ and $\hat{n}_{\lambda}$ is the corresponding number operator.
Inside the cavity the Hamiltonian $H_{t}$ is modified by the replacements
$\hat{c}_{i\alpha\sigma}^{\dagger}\hat{c}_{i\beta\sigma}\to
e^{i\phi_{ij}}\hat{c}_{i\alpha\sigma}^{\dagger}\hat{c}_{i\beta\sigma}$, where
the Peierls phases are $\phi_{ij}=(ea/\hbar){\bf d}_{ij}\cdot\hat{\bf A}$,
${\bf d}_{ij}={\bf r}_{j}-{\bf r}_{i}$ is the vector between atomic sites $i$
and $j$ measured in units of the Ru-Ru distance $a$, and the quantum vector
potential is

$\displaystyle\hat{\bf A}=\sum_{\lambda}(A_{\lambda}{\bf
e}_{\lambda}\hat{a}_{\lambda}^{\dagger}+A_{\lambda}^{*}{\bf
e}^{*}_{\lambda}\hat{a}_{\lambda}).$ (6)

For a given mode $\lambda$ this defines the light-matter coupling
$g_{\lambda}=(ea/\hbar)A_{\lambda}=ea/\sqrt{2\epsilon_{0}\hbar\Omega_{\lambda}V}$.
Thus, the Peierls phases can be written as $\phi_{ij}={\bf
d}_{ij}\cdot\hat{\bf A}$ with

$\displaystyle\hat{\bf A}=\sum_{\lambda}(g_{\lambda}{\bf
e}_{\lambda}\hat{a}_{\lambda}^{\dagger}+g_{\lambda}^{*}{\bf
e}^{*}_{\lambda}\hat{a}_{\lambda}),$ (7)

and the modified hopping Hamiltonian is

$\displaystyle H_{t}=\sum_{\langle
ij\rangle\sigma}\bigg{[}\Big{(}\hat{c}_{iyz\sigma}^{\dagger}\,\hat{c}_{ixz\sigma}^{\dagger}\,\hat{c}_{ixy\sigma}^{\dagger}\Big{)}e^{i{\bf
d}_{ij}\cdot\hat{\bf A}}\begin{pmatrix}t_{1}&t_{2}&t_{4}\\\
t_{2}&t_{1}&t_{4}\\\
t_{4}&t_{4}&t_{3}\end{pmatrix}\begin{pmatrix}\hat{c}_{iyz\sigma}\\\
\hat{c}_{ixz\sigma}\\\ \hat{c}_{ixy\sigma}\end{pmatrix}$
$\displaystyle+t_{pd}\big{(}e^{i{\bf d}_{i^{\prime}i}\cdot\hat{\bf
A}}\hat{p}_{i^{\prime}\sigma}^{\dagger}\hat{c}_{ixz\sigma}+e^{i{\bf
d}_{ji^{\prime}}\cdot\hat{\bf
A}}\hat{c}_{jyz\sigma}^{\dagger}\hat{p}_{i^{\prime}\sigma}$
$\displaystyle\hskip 15.6491pt+e^{i{\bf d}_{j^{\prime}i}\cdot\hat{\bf
A}}\hat{p}_{j^{\prime}\sigma}^{\dagger}\hat{c}_{iyz\sigma}+e^{i{\bf
d}_{jj^{\prime}}\cdot\hat{\bf
A}}\hat{c}_{jxz\sigma}^{\dagger}\hat{p}_{j^{\prime}\sigma}\big{)}+H.c.\bigg{]}.$
(8)

### Hamiltonian in the photon number basis

The Hamiltonian is expanded in the photon number basis $|{\bf
n}\rangle=|n_{1},n_{2},\ldots,n_{N}\rangle$ according to Sentef2020 ; Li2020 ;
Li2021

$\displaystyle H$ $\displaystyle=\sum_{\bf nm}({\bf 1}_{e}\otimes|{\bf
n}\rangle\langle{\bf n}|)H({\bf 1}_{e}\otimes|{\bf m}\rangle\langle{\bf m}|)$
$\displaystyle=\sum_{\bf nm}H_{\bf nm}\otimes|{\bf n}\rangle\langle{\bf m}|.$
(9)

Here ${\bf 1}_{e}$ is the identity operator in the electronic Hilbert space,
and the Hamiltonian $H_{\bf nm}$ is given by

$\displaystyle H_{\bf nm}$
$\displaystyle=\big{(}H_{U}+\sum_{\lambda}\hbar\Omega_{\lambda}n_{\lambda}\big{)}\delta_{\bf
nm}+H_{t,\bf nm}.$ (10)

The matrix elements $\langle{\bf n}|e^{i{\bf d}_{ij}\cdot\hat{\bf A}}|{\bf
m}\rangle$ are obtained by noting that since
$[\hat{a}_{\lambda}^{\dagger},\hat{a}_{\lambda^{\prime}}]=0$, the Peierls
phases factorize over different modes and

$\displaystyle\langle{\bf n}|e^{i{\bf d}_{ij}\cdot\hat{\bf A}}|{\bf
m}\rangle=\prod_{\lambda}\langle n_{\lambda}|e^{i{\bf d}_{ij}\cdot\hat{\bf
A}_{\lambda}}|m_{\lambda}\rangle,$ (11)

where $\hat{\bf A}_{\lambda}=g_{\lambda}{\bf
e}_{\lambda}\hat{a}_{\lambda}^{\dagger}+g_{\lambda}^{*}{\bf
e}^{*}_{\lambda}\hat{a}_{\lambda}$.

The single-mode expressions are calculated by introducing the variables
$\eta_{ij\lambda}=g_{\lambda}({\bf d}_{ij}\cdot{\bf e}_{\lambda})$. Using the
Baker-Hausdorff formula to expand the exponential, the matrix elements are
$\langle n_{\lambda}|e^{i{\bf d}_{ij}\cdot\hat{\bf
A}_{\lambda}}|m_{\lambda}\rangle=i^{|n_{\lambda}-m_{\lambda}|}j_{n_{\lambda},m_{\lambda}}^{ij}$
where

$\displaystyle j_{n_{\lambda},m_{\lambda}}^{ij}$
$\displaystyle=e^{-|\eta_{ij\lambda}|^{2}/2}\hskip 103.85237pt\text{if}\hskip
5.69046ptn_{\lambda}\geq m_{\lambda}$ $\displaystyle\times$
$\displaystyle\sum_{k=0}^{m_{\lambda}}\frac{(-1)^{k}|\eta_{ij\lambda}|^{2k}(\eta_{ij\lambda})^{n_{\lambda}-m_{\lambda}}}{k!(k+n_{\lambda}-m_{\lambda})!}\sqrt{\frac{n_{\lambda}!}{m_{\lambda}!}}\frac{m_{\lambda}!}{(m_{\lambda}-k)!}$
(12) $\displaystyle j_{n_{\lambda},m_{\lambda}}^{ij}$
$\displaystyle=e^{-|\eta_{ij\lambda}|^{2}/2}\hskip 103.85237pt\text{if}\hskip
5.69046ptm_{\lambda}\geq n_{\lambda}$
$\displaystyle\times\sum_{k=0}^{n_{\lambda}}\frac{(-1)^{k}|\eta_{ij\lambda}|^{2k}(\eta_{ij\lambda}^{*})^{m_{\lambda}-n_{\lambda}}}{k!(k+m_{\lambda}-n_{\lambda})!}\sqrt{\frac{m_{\lambda}!}{n_{\lambda}!}}\frac{n_{\lambda}!}{(n_{\lambda}-k)!}.$
(13)

Using the notation ${\bf g}=\\{g_{1},g_{2},\ldots,g_{N}\\}$ to denote the full
set of light-matter couplings, the light-matter coupling is described by the
function

$\displaystyle J_{\bf nm}^{ij}({\bf g})=\langle{\bf n}|e^{i{\bf
d}_{ij}\cdot\hat{\bf A}}|{\bf
m}\rangle=\prod_{\lambda}j_{n_{\lambda},m_{\lambda}}^{ij}.$ (14)

With this function, the hopping Hamiltonian in presence of the cavity can be
written as

$\displaystyle H_{t,\bf nm}$ $\displaystyle=\sum_{\langle
ij\rangle\sigma}\bigg{[}\Big{(}\hat{c}_{iyz\sigma}^{\dagger}\,\hat{c}_{ixz\sigma}^{\dagger}\,\hat{c}_{ixy\sigma}^{\dagger}\Big{)}$
$\displaystyle\times J_{\bf nm}^{ij}({\bf
g})\begin{pmatrix}t_{1}&t_{2}&t_{4}\\\ t_{2}&t_{1}&t_{4}\\\
t_{4}&t_{4}&t_{3}\end{pmatrix}\begin{pmatrix}\hat{c}_{iyz\sigma}\\\
\hat{c}_{ixz\sigma}\\\ \hat{c}_{ixy\sigma}\end{pmatrix}$
$\displaystyle+t_{pd}\big{(}J_{\bf nm}^{i^{\prime}i}({\bf
g})\hat{p}_{i^{\prime}\sigma}^{\dagger}\hat{c}_{ixz\sigma}+J_{\bf
nm}^{ji^{\prime}}({\bf
g})\hat{c}_{jyz\sigma}^{\dagger}\hat{p}_{i^{\prime}\sigma}$ (15)
$\displaystyle\hskip 15.6491pt+J_{\bf nm}^{j^{\prime}i}({\bf
g})\hat{p}_{j^{\prime}\sigma}^{\dagger}\hat{c}_{iyz\sigma}+J_{\bf
nm}^{jj^{\prime}}({\bf
g})\hat{c}_{jxz\sigma}^{\dagger}\hat{p}_{j^{\prime}\sigma}\big{)}+H.c.\bigg{]}.$

### Single-mode light-matter coupling and polarization dependence

For a single photon mode the index $\lambda$ can be dropped, and the function
$J_{\bf nm}^{ij}({\bf g})=J_{nm}^{ij}(g)=j_{n,m}^{ij}$. Writing the intersite
vector on a given bond as ${\bf d}_{ij}=d(\cos\theta_{ij}\hat{\bf
x}+\sin\theta_{ij}\hat{\bf y})$, the parameter $\eta_{ij}$ is given for a
circularly polarized field by $\eta_{ij}=g({\bf d}_{ij}\cdot{\bf e})=gde^{\pm
i\theta_{ij}}$, where $\pm$ denotes a left-handed/right-handed polarization.
Noting that $|\eta_{ij}|=|\eta|=gd$ so that the spatial dependence of the
function $j_{n,m}^{ij}$ factorizes, we have $j_{n,m}^{ij}=j_{n,m}e^{\pm
i(n-m)\theta_{ij}}$ with

$\displaystyle j_{n,m}$
$\displaystyle=e^{-|\eta|^{2}/2}\sum_{k=0}^{m}\frac{|\eta|^{2k+n-m}}{k!(k+n-m)!}\sqrt{\frac{n!}{m!}}\frac{m!}{(m-k)!}$
(16)

for $n\geq m$ and similarly but with $n\leftrightarrow m$ for $m\geq n$.

For linear polarization the polarization vector can be written as ${\bf
e}=\cos\phi\hat{\bf x}+\sin\phi\hat{\bf y}$, and we have
$\eta_{ij}=gd\cos(\theta_{ij}-\phi)$.

### Connection to Floquet Hamiltonian

As a consistency check, the Hamiltonian $H_{t,\bf nm}$ for a single circularly
polarized photon mode is shown to reduce to the Floquet Hamiltonian of Ref.
Sriram2022  in the semi-classical limit. For a single photon mode, we can drop
the mode index, and the function $J_{\bf nm}^{ij}({\bf
g})=J_{nm}^{ij}(g)=j_{n,m}^{ij}$. Taking the polarization vector of the photon
mode to be ${\bf e}=(\hat{\bf x}\pm i\hat{\bf y})/\sqrt{2}$, and writing ${\bf
d}_{ij}=d(\cos\theta_{ij}\hat{\bf x}+\sin\theta_{ij}\hat{\bf y})$, we have
$\eta_{ij}=g({\bf d}_{ij}\cdot{\bf e})=gde^{\pm i\theta_{ij}}$.

To establish the connection to the Floquet Hamiltonian, what remains is now to
show that the function $j_{n,m}(g)$ reduces to the Bessel function
$J_{n-m}(A)$ in the semi-classical limit. To do this, we write $n=m+l$ and
take the limit $m\to\infty$ and $g\to 0$ while keeping $A=2gd\sqrt{m}$
constant. Noting that

$\displaystyle j_{m+l,m}$
$\displaystyle=e^{-|\eta|^{2}/2}\sum_{k=0}^{m}\frac{|\eta|^{2k+l}}{k!(k+l)!}\sqrt{\frac{(m+l)!}{m!}}\frac{m!}{(m-k)!}$
$\displaystyle=e^{-A^{2}/8n}\sum_{k=0}^{m}\frac{\left(\frac{A}{2}\right)^{2k+l}}{k!(k+l)!}\sqrt{\frac{(m+l)!}{m!m^{l}}}\frac{m!}{(m-k)!m^{k}},$
(17)

and that $(m+l)!/m!\to m^{l}$ and $m/(m-k)!\to m^{k}$ as $m\to\infty$, we have

$\displaystyle\lim_{m\to\infty}j_{m+l,m}$
$\displaystyle=\sum_{k=0}^{\infty}\frac{(-1)^{k}\left(\frac{A}{2}\right)^{2k+l}}{k!(k+l)!}=J_{l}(A).$
(18)

Since $l=n-m$ the Hamiltonian $H_{t,\bf nm}$ reproduces the correct Floquet
Hamiltonian in the semi-classical limit.

### Cavity-modified spin Hamiltonian

The total Hamiltonian $H$ is down-folded to the spin sector by using quasi-
degenerate perturbation theory to eliminate $H_{t}$ order by order.
Winkler2003  This perturbation expansion is performed numerically up to fourth
order in $t/U$ using exact diagonalization of the Ru2Cl2 cluster shown in Fig.
3e. This allows to extract the nearest-neighbor magnetic interactions and
their dependence on the light-matter coupling, cavity frequency and
polarization. The structure of the perturbation expansion allows the down-
folding to be performed separately within each photon sector, giving a coupled
spin-photon Hamiltonian of the form

$\displaystyle\mathcal{H}$ $\displaystyle=\sum_{\bf
nm}\big{(}\mathcal{H}_{s,\bf nm}+\delta_{\bf
nm}\sum_{\lambda}\hbar\Omega_{\lambda}n_{\lambda}\big{)}|{\bf
n}\rangle\langle{\bf m}|.$ (19)

Here $\mathcal{H}_{s,\bf nm}$ is the spin Hamiltonian in the photon sector
connecting photon numbers $\bf n$ and $\bf m$, and is given by

$\displaystyle\mathcal{H}_{s,\bf nm}$ $\displaystyle=\sum_{\langle
ij\rangle}\begin{pmatrix}S_{i}^{\alpha}&S_{i}^{\beta}&S_{i}^{\gamma}\end{pmatrix}\begin{pmatrix}J&\Gamma&\Gamma^{\prime}\\\
\Gamma&J&\Gamma^{\prime}\\\
\Gamma^{\prime}&\Gamma^{\prime}&J+K\end{pmatrix}_{\bf
nm}\begin{pmatrix}S_{j}^{\alpha}\\\ S_{j}^{\beta}\\\
S_{j}^{\gamma}\end{pmatrix}$ $\displaystyle+B_{\bf nm}\sum_{i}\hat{\bf
e}_{B}\cdot{\bf S}_{i}$ (20)

where each bond $(ij)$ is labeled by the indexes
$\alpha\beta(\gamma)\in\\{xy(z),yz(x),zx(y)\\}$. In this Hamiltonian, the
magnetic parameters $J$, $K$ and $\Gamma$ all depend on the light-matter
coupling ${\bf g}$ and the photon numbers ${\bf n}$ and ${\bf m}$.

### Exact diagonalization

The ground state of the coupled spin-photon system was obtained by exact
diagonalization of a 24 site spin cluster interacting with a single photon
mode. To perform the calculations we used to open-source Python package
QuSpin. Weinberg2017 ; Weinberg2019

Figure 4: Electronic bandstructure of $\alpha$-RuCl3. Highest valence bands
(red) and lowest conductions bands (blue) of $\alpha$-RuCl3 obtained within
the DFT+$U$ formalism. TancogneDejean2017

### Model parameters from first principles

To determine the parameters of the electronic Hamiltonians $H_{U}$ and
$H_{t}$, we performed first principles simulations of monolayer $\alpha$-RuCl3
with the Octopus electronic structure code. TancogneDejean2017 ;
TancogneDejean2020  The single particle parameters where obtained from a
Wannierization of the Ru $4d$ and Cl $3p$ orbitals in the paramagnetic state
using Wannier90, Marzari97 ; Souza01  while the interaction parameters where
determined using the hybrid DFT+$U$ functional ACBN0 in the zigzag state.
TancogneDejean2017  We have checked that the single particle parameters differ
by less than one percent between the paramagnetic and zigzag states. The
resulting electronic parameters are given in Tab. 1, the equilibrium spin
parameters in Tab. 2, and the electronic band structure is shown in Fig. 4.

The calculations were performed in a $1\times\sqrt{3}$ supercell to account
for the zigzag magnetic structure, using the experimental lattice parameters
$a=5.98$ Å and $b=10.35$ Å. Mixed boundary conditions, periodic in the in-
plane direction and open in the out-of-plane direction, where used together
with a vacuum region of $15$ Å to ensure convergence in the out-of-plane
direction. A $8\times 8$ $k$-point grid and a real-space grid spacing of $0.3$
Bohr were employed. Using the ACBN0 functional a self-consistent effective
interaction $U_{\rm eff}=U-J_{H}$ was determined on both the Ru and Cl ions,
and the Kanamori parameters $U$ and $J_{H}$ where calculated in the final
states after convergence had been reached.

Parameter | $U$ | $J_{H}$ | $\lambda$ | $\Delta_{pd}$ | $t_{1}$ | $t_{2}$ | $t_{3}$ | $t_{4}$ | $t_{pd}$
---|---|---|---|---|---|---|---|---|---
eV | 3.66 | 0.64 | 0.18 | 4.34 | 0.035 | 0.035 | -0.06 | -0.023 | -0.8
Table 1: Electronic parameters of $\alpha$-RuCl3: Parameters of the Hubbard-Kanamori Hamiltonian $H_{U}$ and the hopping Hamiltonian $H_{t}$ as calculated from first principles with the Octopus and Wannier90 electronic structure codes. Parameter | $J$ | $K$ | $\Gamma$ | $\Gamma^{\prime}$
---|---|---|---|---
meV | -0.078 | -7.91 | 2.96 | -0.90
Table 2: Equilibrium spin parameters of $\alpha$-RuCl3: Equilibrium spin
parameters of $\alpha$-RuCl3 for the model $\mathcal{H}_{s}$, calculated from
the electronic parameters of Table 1.

For a two-dimensional cavity the lowest energy photon mode has a frequency
$\Omega=\pi c/L_{z}$, where $L_{z}$ is the extent of the cavity in the
$z$-direction. Therefore, the light-matter coupling is independent of the
photon frequency, and is given by

$\displaystyle g=\frac{ea}{\sqrt{2\epsilon_{0}\hbar\Omega
a_{x}a_{y}L_{z}}}=\frac{ea}{\sqrt{2\pi\epsilon_{0}\hbar ca_{x}a_{y}}}=0.12$
(21)

assuming $a_{x}a_{y}=a^{2}$.

## Data Availability

All data supporting the findings of this study are available from the
corresponding authors upon reasonable request.

## Code Availability

The codes used to generate the data of this study are available from the
corresponding authors upon reasonable request.

## Acknowledgements

We acknowledge support by the Max Planck Institute New York City Center for
Non-Equilibrium Quantum Phenomena, the Cluster of Excellence ’CUI: Advanced
Imaging of Matter’- EXC 2056 - project ID 390715994 and SFB-925 ”Light induced
dynamics and control of correlated quantum systems” – project 170620586 of the
Deutsche Forschungsgemeinschaft (DFG), and Grupos Consolidados (IT1453-22).
The Flatiron Institute is a Division of the Simons Foundation.

## Author Contributions

## Competing Interests

The authors declare no competing financial or non-financial interests.

Figure S1: Polarization dependence of the Kitaev interaction for a linearly
polarized cavity mode. Effective Kitaev interaction $\bar{K}$ as a function of
the effective light-matter coupling $\bar{g}=2g\sqrt{n_{\rm av}}$ and
frequency $\Omega$ for a cavity with $n_{\rm av}=1$. The different panels
correspond to polarization vectors making and angle $\phi$ with the $x$-axis.
Due to the $C_{3}$ symmetry of the magnetic system, the polarization
dependence shows a period of $\phi_{0}=2\pi/3$. Figure S2: Photon occupation
of an electronic cluster. Photon occupation of the ground state of a four site
Ru cluster described by the Hamiltonian in Eqs. Effective low-energy lattice
electron Hamiltonian and Hamiltonian in the photon number basis. The Ru
cluster is schematically shown to the right. Figure S3: Spin parameters of the
dark cavity. Normalized magnetic exchange interaction $\bar{J}$, Kitaev
interaction $\bar{K}$, and anisotropy interactions $\bar{\Gamma}$ and
$\bar{\Gamma}^{\prime}$ as a function of the light-matter coupling $g$ and the
photon energy $\hbar\Omega$ in the zero photon sector $n_{\rm av}=0$. Figure
S4: Spin parameters of the seeded cavity. Normalized magnetic exchange
interaction $\bar{J}$, Kitaev interaction $\bar{K}$, anisotropy interactions
$\bar{\Gamma}$ and $\bar{\Gamma}^{\prime}$ and induced magnetic field
$\bar{B}$ as a function of the effective light-matter coupling $\bar{g}$ and
the photon energy $\hbar\Omega$ in the zero photon sector $n_{\rm av}=1$.